DOCUMENT RESUME 



ED 043 296 



24 



HE 001 76 



AUTHOR 

TITLE 

INSTITUTION 



SPONS AGENCY 



BUREAU NO 
PUB DAT? 

NOTE 

AVAILABLE FROM 



Lawrence, Ben, Ed.; And Others 

Outputs of Higher Education: Their Identification, 
Measurement, and Evaluation 

American Council on Education, Washinoton, D.C.; 
California Univ., Berkeley* Center for Research and 
Development, in Higher Education.; Western Interstate 
Commission for Higher Education, Boulder, Colo. 

Esso Education foundation.; National Science 
foundation, Washington, D.C. : Office of Education 
(PHEW), Washington, L,C. Bureau of Research. 
PP-8-070B 
Jul 70 

1 2?p , • Parer s from a seminar, Washinoton D.C., Mav 
3-5, 1970 

Western Interstate Commission for Fioher r ducation, 
P.0. Drawer P. , Boulder, Colorado 80302 (S3. 50) 



EDPS PRICE EDPS Price ME-50.50 HC-J6.45 

DESCRIPTORS Educational Needs, *Educational Objectives, 

*Fvaluation, *Hioher Education, *Mea sure ment , ^ 

Measurement Techniques, systems Analysis 



ABSTRACT 

The purpose of the seminar was to find some answers 
to, and a deeper understanding of the followina auestions. (1) What 
is the purnose or are the purposes of hiqher education? (?) what are 
the benefits of hioher education to the individual and to society? 

(?) Can the outputs of higher education be named and measured? (4) 
Where is hioher education going, where do ve wa it it to go, and where 
does it need to go? Several issues were identified: (1) the summons 

to accountability by students, society, and the legislature; (?) 
flanging financial support; and (3) higher education's earlier 
overreactior to short-term needs. The papers in this ;>uHicition deal 
oenerally with the problem of output. Paoers by F. F. Palderston and 
John Vaizey survey the problem. The conceptual analysis of output is 
dealt with bv David g. Brown, C. vest Churchman, Alain C. Fnthoven, 
and Kenneth S. Tollett. Fome approaches to measurement of output are 
suggested by Alexander V. Astin, John E. Brandt, Potbin R. Hough, and 
John P. Miller. A bibliography and a list of participants are 
included. (AE) 




EDO 43296 



&* ?- * 7*8 



OUTPUTS OF HIGHER EDUCATION: 

Their Identification^ Measurement, and Evaluation 



Papers from a seminar held at Washington, D. C., 
May 3*5, 1970, conducted by the 
Western Interstate Commission for Higher Education 
in cooperation with the 
American Council on Education 
and the 

Center for Research and Development 
In Higher Education at Berkeley. 



Edited by 

Ben Lawrence, Qeorge Weathersby, and Virginia W. Patterson 



The seminar was supported by 
the U. 8. Office of Education, 
the ESSO Foundation, 
and the National Science Foundation 



Western Interstate Commission for Higher Education 
Post Office Drawer P Boulder, Colorado 10302 

July 1970 

AMMomi te p tee ereneWe, fS.SC. 




I 






@TO/DC§K]S 



Executive Director: WICHE 
Robert H. Kroepsch 

Associate Director, WICHE, and 
Director, Planning and Management 
Systems Division: 

Ben Lawrence 



Program Director for Development and 
Applications Unit: 

Warren W Gulko 

Program Director for Training Unit: 

Robert Huff 

Program Associate for Information System!: 
Charles R. Thomas 



V\ 




The Western Interstate Commission for Higher Education 
(WICHE) is a public agency through which the 13 western 
states work together 

. . . to increase educational opportunities for westen>eri. 
... to expand the supply of specialized manpower In the 
West. 

... to help universities and colleges improve both their 
programs and their management. 

... to inform the public about the needs of higher educa- 
lion. 



The program of the WICHE Planning and Management 
Systems Division was proposed by state coordinating agen- 
cies and colleges and universities in the West to be under 
the aegis of the Western Interstate Commission for Higher 
Education. The Planning and Management Systems Di- 
vision program proposes in summary: 

To design, develop, and encourage the Implementation of 
management information systems and data bases including 
common data elements in institutions and agencies of higher 
education that will: 

• provide improved information to higher education ad- 
ministration at all levels. 

• facilitate ctchange of comparable data among institu- 
tions. 

• facilitate reporting of comparable information at the 
state and national levels. 
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The Western Interstate Commission tor Higher Education, the American 
Council on Education, and the Center for Research and Development in Higher 
Education at the University of California, Berkeley, present the papers of a national 
research trailing seminar on the outputs of higher education held in Washington, 
D. C., May 3*5, 1970. This seminar was jointly supported by grants from the 
Bureau of Educational Research of (he United States Office of Education, the 
ESSO Educational Foundation, and the National Science Foundation. 

The quality, effectiveness, and costs of higher education arc discussed and 
much debated today. While the resources required to iun an institution of higher 
learning have been described and accounted for in detail, very little agreement is 
to be found on the measured benefits to society and to the individual from the 
use of those resources. 

Planning to accommodate (he sheer volume growth in the educational enter- 
prise is a demanding (ask in itself. Persuasion of the need to finance that growing 
enterprise is a still more difficult task and requires better information than is 
presently available. 

We need more specific and more meaningful descriptions of the entire process 
of higher education. We need to see resource inputs, activities in the educational 
setting, and outputs of higher education as related parts of a whole. We are looking 
for insight and understanding of just how the contributions, activities, and bene- 
fits of higher education may be shaped, modified, directed, and improved through 
intelligent decision-making and informed allocation of resources. 

With these concerns in view, this research training seminar brought together 
administrators active in higher education, students, legislators, researchers, and 
quantitative analysts who have been actively concerned with these problems. These 
individuals reviewed and commented upon prepared papers and discussed imme- 
diate, critical questions 

We gratefu'ly acknowledge the contribution of the authors of these papers; 
the seminar commentators; the invited speakers; and the staffs of the Council, 
Center, and WICHE for planning and conducting the seminar. 
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INTRODUCTION: Now Where Do We go? 



“Our mandate is clear ... We are going to 
have to prove that we deserve the dollars spent on 
higher education and justify our asking for each 
additional dollar," says an educator. 

What is the function of the university com- 
munity in a free society ordered by law? Our 
notion of university as an ivory tower gave way 
to the idea of university as an open marketplace 
for ideas. Recent campus activities seem io suggest 
that the image of university as marketplace may 
be replaced with the concept of university as an 
instrument of social change. 

What are the purposes of higher education? Is 
the university's primary responsibility to society? 
Or to the students and faculty engaged in the 
academic enterprise? Docs the structure of the 
university serve the purposes of the university? 
Are the resources of the university committed to 
its purposes? 

Self examination has been forced upon our 
institutions of higher education by alienated stu- 
dents. disaffected faculty, dissatisfied legislators, 
disenchanted alumni, and disappointed parents 
who are challenging the university. 

Thorny as these questions may be, they are 
being asked. They are being shouted. And the 
answers are being subjected to unprecedented 
scrutiny and debate. Both internally and externally, 
the university is being called to accountability. 

To the problem of accountability, add the dif- 
ficult problem of financing higher education. 
Spiraling costs, increased enrollments, and light 
money compound the fiscal crisis which confronts 
university decision-makers. Money spent to hire 
more teachers is not available to improve salary of 
existing faculty. Money spent to beef up the engi- 
neering program is not available to improve cur- 
ricular offerings In the fine arts. Money spent to 
equip offices of added personnel is not available 
to buy bookcases and files in existing staff quarters. 

The modern university decision-maker is 
swamped with problems, files of facts, and moun- 
tains of requests. So much has been said, recorded, 
published, televised, and proclaimed about the 
academic community that, certainly, there h no 
lack cf raw* data. But presently the man who must 
make decisions for his institution is no; given much 
help. Despite the abundance of facts, they are not 
sufficiently useful. They have not Veen selected 
and aggregated in manageable form. 



Accumulation and analysis of information will 
never replace judgment. Only the decision-maker 
can reflect the priori. ies established for his insti- 
tution. But (here is no doubt that more meaningful 
information would assist the man-on-the-spot as 
he faces tough choices: 

Does establishment of the new degree program 
in the graduate school mean (hat (he undergradu- 
ate program in the same department must simply 
mark time for another year? 

Will the extra cost cf tutorial services for mi- 
nority students mean that hiring counselors for in- 
coming freshmen will have to be postponed? 

Much more is demanded of the college presi- 
dent than the academic credentials and experience 
noted on a job Ascription. Horse sense, persua- 
sion, and the gift of prophecy are valued requisites. 
In addition to what is expected of him as a public 
figure, he is charged with the responsibility of 
maintaining rapport with his faculty through dis- 
plays of scholarly attainment, savvy with the legis- 
lature, and skills of academic governance. With stu- 
dents he must evidence courage, candor, and con- 
science. 

Small wonder (hat (he man of whom so much 
is asked is beginning to ask for help himself. He is 
looking for insight and understanding of just how 
the programs of higher education at his institution 
may be shaped, modified, directed, and improved 
through informed decision-making and intelligent 
allocation of resources. 

In search of meaning and understanding of the 
process of higher education, educators, administra- 
tors. students, legislators, and analysis represent- 
ing various types of institutions and geographic 
areas knocked heads at the WICHE MIS Seminar, 
"Outputs of Higher Education— Their Identifica- 
tion. Measurement, and Evaluation," 

Initially there was little agreement within the 
group. Their task was enormous. Each man con- 
tributed to the discussion according to his own 
experience. Gradually, a form of consensus 
emrrged on the issues which have produced the 
present problem. Then discussion turned to the 
substantive considerations of the seminar, but 
consensus was not reached on answers to the 
cosmic questions raised: 

What Is the purpose of higher education? Is 
there more than one? If so, what are they? 
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What are the benefits of higher education? To 
the individual? To society? 

Can we name the outputs of higher education? 
If we can name them, can we measure them? How? 

Where are we going? Where do wc want to 
go? Where do we need to go? 

It was characteristic of seminar participants 
that each question received full debate. There were 
no sacred cows at this meeting. Divergent ideas 
and perspectives were heard and considered. While 
no general consensus emerged from the reasoned 
discussion, a far more precise picture of the task 
ahead developed. 

The agony of considering this problem did not 
remove the need for con'inuing to attack it! 

The iMuee Identified 

8umnu>ft» to AccountaMHty 

Something has happened on the American 
campus in the last decade, something has happened 
that has never occurred before in the two hundred 
year history of higher education in our country. 
A set of circumstances and conflicting interests met 
head on and brought the campus scene into sharp- 
er public focus than ever before. 

Consider the actors on stage in this drama of 
discord and confrontation. 

The student finds much of the teaching in his 
classes irrelevant to the world he sees, unresponsive 
to his needs and to the needs of society. Enter stu- 
dent activism. Demonstrations, demands, and po- 
litical activity arc the order of the day. For ob- 
'.elvers of the campus scene, the panty raids, gold- 
fish-swallowing contests, and phone booth-cram- 
ming marathons of the past acquired a nostalgic 
aura ol approval of college high jinks as they 
"ought" to be, compared with the political acti- 
vism of the present. The new generation of col- 
legians not only questioned the legitimacy of cur- 
ricula, but questioned the very purpoi? of the uni- 
versity, demanding a voice in determining degree 
requirements and use of time in the academic com- 
munity. Students wanted a voice in determining 
every aspect of campus environment and student 
life. The notion of university acting in toco parentis 
w-as first to go. Then, tail homecoming parades, 
school spirit, the fun and frolic of campus life. 
Farewell the bonds of group associations. Bye-bye 
the automatic respect accorded a professor. 



Up! the student individual. 

Having implored generations of students at 
graduation to go forth into the world, questing to 
make it a better place, the facul’ics of our colleges 
should not have been surprised when collegians ac- 
cepted the challenge. Trouble is, students jumped 
the gun and decided not to wait till graduation to 
improve the world. They included the university in 
the world and wanted to “improve" it. too. Admis- 
sions policies, grading, examinations, the leclur: 
system came under student fire. When a moral 
crusade is waged away from home, it is easy to 
support. Aimed directly at you. a crusade becomes 
threatening, no matter the validity of its cause. A 
growing rumble arose from the faculty row-, "Just 
what are these kids here for?" 

Heard off stage: Mutters. Grvmbles. Then, 
outraged cries of distress from the parents of the 
present collegiate generation. Parents first, they 
;.'c taxpayers second. Spiraling costs of living vie 
for attention with increasing unemployment. Fed- 
eral income taxes arc taking a big bite and there 
seems to be no relief in sight. Americans arc going 
to Vietnam and Americans arc going to the Moon, 
but Mr. Middle America is going no place. Instead 
of his annual vacation trip, he slays home and 
wotks on his yard. Frustrated by rising expecta- 
tions. strapped for funds, he looks to his state legis- 
lature for a little help on tuition for his youngster 
attending ar in-state college. And what happens? 
Tuition increases, up and down the line. 

Now, take a look at the state legislator. His 
judgment and vote become the referees in a rough 
and tumbie competition for the public dollar. Ev- 
erybody wants public money for a pet project. 
Health and welfare programs require a fresh in- 
fusion of state funds. Junior colleges arc continuing 
to spring up. Former junior colleges arc becoming 
four-year, baccalaureate, degree-granting institu- 
tions. Nvvv dormitories arc needed on every cam- 
pus — and more classroom space, more mon:y for 
salaries for more faculty members! As one edu- 
cator pointed out. "The problem has been dearly 
put. It*$ mental health against education, education 
against welfare, housing against education, and the 
problem is that every one of these goals is good. 
Scwne are more immediate than others, but who is 
to say that investing in housing is wrong?” 

Each member of a state legislature has a diffi- 
cult job to do. Mindful of the squeeze on his con- 
stituents between heavy federal taxes and the in- 
creasing costs of paying for primary and secondary 
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education through a property lax. he is prudent and 
watchful of expenditures at the stale level. With all 
the clamor and competition for the slate dollar, he 
is skeptical when he is asked lo overlook — or look 
beyond — the chaos which lie observes on campus 
and vote dollars in support of higher education al 
that institution. Already sensitive to the problem, 
just one more demonstration, one more insult to 
an invited visiting dignitary, or one more faculty 
protest about a loyally oath and he finds himself 
asking (and being asked). "Is this what we arc 
spending money for?" 

As never before, the university is being asked 
lo justify itself — its purpose, its methods of 
achieving that purpose; its allocation of precious 
-esourc a s; ils priorities: its responsibilities to the 
individual and to society. Yes. both from within 
and from without, institutions of higher education 
arc being called to account. 

Flagging Financial Support 

In the past decade higher education witnessed 
a growing reluctance on the part of state, federal, 
and private sources to finance iis operations. 
Granted the impact of competing demands for the 
same dollar, granted the effect of an inflationaiy 
spiral of costs, perhaps the single most significant 
factor contributing lo the lagging financial support 
of higher education was the growing public disaf- 
fcction with campus disorders and with the objec- 
tives of higher education, as they were perceived by 
the public. With intellectually elite young people 
earning entrance to a university, legislators are dis- 
gruntled to see the universities admitting only the 
cream of the intellectual crop and then not chal- 
lenging these academically elite students to do some- 
thing with their lives. Summing up this feeling, one 
legislator said. "Legislators arc concerned and 
Mghtened by the doctrine of elitism which sa>s 
thal. because you hast; selected the elite, they should 
be canonized. If you have tlwl kind of school you 
have a right lo be concerned. We want you to 
manufacture elites, not canonize existing elites on 
entry*. If they are just going lo mark lime for four 
years. O.K. Let them live it out. But don't expect 
public money! In fact, if you do not challenge stu- 
dents to do something productive with the years 
spent in college, expect cowboy wrath. We are go- 
ing to close them out!" 

Not only politically, but financially, higher edu- 
cation stands in the jaw^s of a vise. On one hand, 
the costs of every component of institutional opera- 
tions are rising — interest rales, salaries, supplies, 
services, insurance. Costs are climbing steadily and 
have already reached historic heights. On the other 



hand, income from all sources is either falling be- 
hind or is increasing at a rate loo stow to keep up 
with burgeoning student enrollments. One alterna- 
tive popular with governing boards is to pass more 
of the cost on to students in the form of tuition in- 
creases. One consequence of this decision is lo 
quicken student realization of the major share of 
the cost of education they arc already bearing and, 
therefore, to step up their demands for a meaning- 
ful role in the governance of institutions of higher 
education. Thus, we hear students saying. “We arc 
paying for this education. Give us a voice In deter- 
mining the kind and quality of the education avail- 
able. What you say about education to appeal to 
the legislature is not what we want to hear." 

Though recognizing the validity of the stnte- 
mcnl. "He who pays the piper calls the lurie." the 
university is caught on the ho. as of a dilemma. 
What do you do when two people who pay the 
same piper call for different tunes? 

Ovarreactlon \o Short-Term Naedi 

Higher education is p "vying the price now for 
an earlier overreaction to the immediacy of a press- 
ing problem. What military disaster or s^ial chaos 
was averted by our national response to the chal- 
lenge of Sputnik, God only know"! 

The problem is one of "overkill." Educators 
were delighted to receive massive federal subsidies 
in the physical sciences and engineering, but that 
delight paled when they noted the distortion in the 
composition of their student bodies and curricula. 
To their dismay, the supply of graduates in those 
fields is no longer responsive to the employment 
needs of business, industry, and the non-defense 
sectors of government. 

Our information systems were not sensitive 
enough to forewarn us of the consequences of a 
narrow emphasis on the technological aspects of 
higher education, of the neglect of the humanities 
which inspire ami temper our values, and of the 
disregard for the forces of the supply and demand 
for graduates in a specialized area. As a result, 
graduates of many fields are currently facing severe 
employment problems. They are directing their 
despair and disillusionment toward their institu- 
tions of higher education. The absence of reliable 
information on employment opportjnilies is a sig- 
nificant source of student dissatisfaction. 

Higher education now faces the problem of 
stimulating thought and activity in that province of 
human endeavor which deals with man and how 
he bas karned to live with other men; how* he ex- 
presses his values; how he perceives beauty, truth. 
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his environment; how he behaves and why; how 
he uses his leisure; how he finds meaning in life, 

Added to the problem of financing higher ed- 
ucation and restoring a balance in curricular of- 
ferings, the university is going to have to be far 
more aware of the demand for jobs that it is pre- 
paring students to fill. A management consultant 
observes, “As we look ahead from this day on, I 
think higher education is going to have to be much 
more concerned about the market for the product. 
It is going to have to be concerned about its 
product mix, I submit that higher education is on 
the threshold of major marketing problems. Now 
we are looking at a lessening of that market and 
this year and, perhaps for several years to come, 
we are going to have unemployed, and quite a 
few underemployed. Ph.D.’s. I do think educators 
owe it to their customers to let them know some- 
thing about the demands of people in various dis- 
ciplines and what some of the rewards, both mone- 
tary anl intangible, are going to be in pursuing the 
various disciplines,” 

One Response to the Demand For 
Accountability — the Use of Activity 
and Output Measures 

By legislators, by hard-pressed taxpayers, by 
a crusading press, universities from coast to coast 
are being told to measure the effectiveness of their 
efforts and the cost required to produce that effec- 
tiveness, “Define quality and then tell me if you 
can deliver it for two cents a ton cheaper,” prods 
a state fiscal analyst. 

“Do it yourself or we will do it for you,” is the 
general promise. 

Understandably resistant to pressures from 
within and without, institutions of higher educa- 
tion arc nevertheless coming to an understanding 
that it is far wiser to be part of the process which 
suggests forms of measurement and identifies the 
areas of measurement. The VVICHE MIS Program 
is built on that belief. 

Now as ever before, educational institutions 
are committed to the use of reason. What, then, 
does the use of management information and an- 
alysis offer the university decision-maker? Man- 
agement information and analysis add a new di- 
mension to reason, providing the power of fact 
and reality to the wishes and aspirations of the 
institution. 

Our current challenge is to focus our collective 
reasoning and analytical abilities on the serious 
problems that are fraying and unraveling the very 



fabric of our institutions. We need to understand 
the roles of various decision-makers as they affect 
higher education — including the roles of students, 
faculty, administration, legislators, government of- 
ficials, community representatives, representatives 
of private industry, alumni, and others. Further- 
more, we need to understand the important at- 
tributes or characteristics of higher education as 
viewed by each of these decision-makers, Under- 
standing and description are important first steps 
to the measurement and evaluation of the activi- 
ties and outputs of higher education. 

Recogn/re Important Attributes 

Before we can measure an activity or an output, 
we need an operational, unambiguous definition of 
that activity or output. If one is analyzing a trans- 
portation company, one might consider relevant at- 
tributes to be the origins and destinations of the 
load, the time required, the cost, the volume or 
dollar value of goods carried, the reliability of serv- 
ice, and the average damage or loss in transit. Un- 
dergraduate education as viewed by the state might 
be characterized by many of the same variables, 
However, students might consider their educational 
experience summarized by their chance of success- 
fully completing their degree program, the propor- 
tion of courses which they may elect, their chance 
of marriage, the level of social concern of the peers, 
the accessibility of faculty members, the cost of at- 
tending the school, and other factors. 

The major point of this discussion is that dif- 
ferent roles have very different perspectives of the 
institution and a different set of descriptive at- 
tributes is appropriate for each decision-making 
role. Therefore, in our analysis of the major chal- 
lenges our institutions face, it is critically impor- 
tant that we identify all of the relevant decision- 
making roles and then choose the attributes ap- 
propriate to each role. For example, if a college or 
university is seeking to increase its service to the 
minority communities, it not only needs to count 
the minority students enrolled and their grade point 
averages, it also needs to look at the school from 
the student's perspective and see’ what kinds of 
social groups and activities have evolved, the de- 
gree of stress and acceptance, the attitudes of ad- 
ministrators. and so forth. 

Measures for Attributes 

To enable each actor in the drama of higher 
education to play his part to the best of his ability, 
we need to provide each individual with personally 
relevant information. However, we need 'o devise 
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appropriate measures for the attributes which char- 
acterize higher education from each major decision- 
making perspective before we can provide the 
requisite information. It is easy to count heads or 
add up grade points, but it is much more difficult 
to develop a measure of student or faculty stress 
or a measure of their sense of control of their 
own environment. At this point, we can draw upon 
experts. 

Evaluation ol Activities and Outputs 

A comprehensive, operational taxonomy of the 
attributes of higher education provides the basis 
for operational definitions and measurement of in- 
stitutional activities and outputs. However, the 
relative values of each activity and output need to 
be expressed before an administrator can con- 
sistently choose between alternative policies or ex- 
penditure patterns. For example, the expenditure 
of an additional $100,000 might enable an institu- 
tion to produce either two additional M.D.’s, or 
four additional Ph.D.'s in physics, or 25 additional 
Ph.D.’s in French, Which path should the adminis- 
trator choose? In other words, how much should the 
administrator value one more doctorate in medi- 
cine, physics, or French? There is no easy answer 
for the administrator — he must turn to his con- 
science, to his constituency, to his advisers, and 
then synthesize a set of priorities that he is willing 
to act upon. However, once he chooses an opera- 
tional set of values or weights, these numbers 
should be included in the activity and output ac- 
counting system and reported to the administrator 
as a guide to action. Therefore, the accounting and 
reporting system based upon thus taxonomy of at- 
tributes and measures should also include the 
framework or mechanism which explicitly incor- 
porates the decision-maker’s values. 



Decision-Making Aided 

The basic assistance to decision-making that 
the use of activity and output measures offers is 
one in which a person, faced with a difficult re- 
source allocation decision, seeks to think through 
his problem very carefully. He first identifies the 
characteristics of higher education that are impor- 
tant to him and then selects an appropriate meas- 
urement technique. By associating relative values 
with each attribute he is then able to identify the 
most desirable course of action. This process does 
not replace or denigrate the position of the decision- 
maker, just as power steering or power brakes do 
not obviate the role of an automobil® driver. How- 
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ever, this approach to decision-making does extend 
the capabilities of the decision-maker, just as pow- 
er steering or power brakes augment a driver's own 
expression of control. Actually, the position of the 
administrator should be enhanced because his un- 
derstanding and control of his system is increased 
and facilitated. 

A major advantage of this approach is that 
institutional objectives can be expressed operation- 
ally in terms of the attributes ascribed by the de- 
cision-maker to his institution. In practice, these 
objectives can be articulated either as specific tar- 
gets or as relative values or inte nsities. For exam- 
ple, instead of saying every student should be able 
to read, we stipulate the specific target that every 
student should score at least 625 in verbal ability 
on the SAT or GRE test. We might indicate that 
verbal ability is twice as important as mathematical 
ability, or vice-versa, and make our resource allo- 
cation decisions in light o: this choice. 

This general approach contributes to the reso- 
lution of intra-institutional conflict by enabling 
conflicting groups to separate a decision problem 
into its elements. The conflict may result from dif- 
ferences over choice of attributes, measures, or 
weights. In either case, the conflicting parties can 
draw a line of demarcation around the areas of 
agreement and concentrate their attention on the 
essence of the dispute. This might rise to create 
new alternatives, separate activities, or some other 
resolution. At least, this approach would map the 
terrain of conflict and identify the problem areas. 

On the technical side, the characterization of 
the different decision-makers’ systems provides the 
basis necessary for various quantitative analyses, 
analytical modeling, or management information 
systems. These analytical developments are hin- 
dered by our inadequate understanding of the na- 
ture, attributes, and problems of decision-makers 
of higher education, rather than any technical lim\ v 
tations. Mathematically elegant solutions are of lit- 
tle value unless we have asked the correct ques- 
tions and applied the appropriate values. 

Roughly Right or PrecUely Wrong? 

However, these advantageous observations 
should be tempered by justifiable criticisms. Some 
observers point out that our quantitative analysis 
techniques and measurement tools are still suf- 
ficiently crude and imprecise as to leave consid- 
erable doubt as to the validity of the measures — 
which is partly true. In response, one person said, 
“I realize that taking the temperature doesn’t prove 
that you are intelligent but taking the temperature 
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shows that you may not be dead.*’ Furthermore, 
in the words of Alain Enthovan, “It is better to be 
roughly right than precisely wrong. 11 

While it seems obvious that serious attempts 
will be made to identify and measure the activities 
and outputs of higher education in the iext several 
years, there are those who seriously challenge 
whether it is a feasible course of action. They 
recognize that some of the outputs of higher edu- 
cation are easily identified and probably can be 
measured. However, they argue, “Rather than be- 
ing merely unsophisticated in the identification and 
evaluation of the units of output, it is possible that 
there are sim iy no original units of output which 
actually exis f for us to measure.” If we cannot 
identify all o 4 the benefits of higher education, how 
big is the * lole of the benefits of higher educa- 
tion? Is it appropriate to assign costs of higher 
education only to those benefits that can be iden- 
tified? 

In response to (his philosophy, the proponents 
of measuring the outputs of higher education indi- 
cate that we do not normally, in the course of 
human events, pay out money for something that 
we cannot identify. When we make a purchase 
which has side benefits, we do not normally change 
our accounting structure to indicate that the bene- 
fit that we were originally purchasing corts us X 
number of dollars l ess because we got a side bene- 
fit with the value of Y dollars, We usually accept 
the windfall as an unplanned bonus. Proponents 
argue that higher education should look upon 
those benefits of higher education progress that 
cannot be identified and measured in the same way. 

Can we attach values to the outputs we meas- 
ure without first having an understanding of the 
purposes of higher education? There are those who 
argue that before we begin the measurement of (he 
outputs of higher education, we ought to develop 
a clear understanding of the purposes of higher 
education. Then and only then would it be pos- 
sible for us to determine a value for the specific 
outputs. For example, a specific output may cost 
$5,000 to produce. It may have very specific quan- 
tifiable characteristics, but if its production is not 
useful to the purpose of the institution, it has no 
value for that purpose. 

Those who urge immediate attention be given 
to measuring the outputs of higher education con- 
cede that values must be placed upon the outputs 
in relationship to the purposes of higher educa- 
tion, On the other hand, they indicate that attempts 
to reach agreement on the general purposes of 
higher education today appear to be fruitless. If 
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we wait for agreement on this course of action we 
may never do anything to improve our muddled 
state of affairs. They further argue that higher edu- 
cation must have some purpose, otherwise it would 
not continue to exist. The purpose of higher edu- 
cation can, from one point of view, be defined 
as the aggregation of those identifiable products or 
outputs of higher education. Looking at it from a 
purely analytical point of view, if we can define 
the outputs and describe the process between the 
inputs and outputs we have, in fact, not only 
described the system, but indicated its purposes 
from an empirical point of view. 

Another issue is at what point we should meas- 
ure the outputs of higher education? Should we at- 
tempt to measure the immediate products of higher 
education or the long-range benefits of higher edu- 
cation to society? Should we attempt to measure 
medical doctors produced or the quality and extent 
of medical services to our people? When looking 
at the institution from what one calls the public 
service point of view, measuring the health services 
provided to our people appears to be a justifiable 
approach. On the other band, those who argue that 
we should measure the immediate products of our 
institutions of higher education point out that many 
factors other than the institutions and process of 
higher education contribute to the effectiveness of 
health services. They say that higher education 
cannot be held accountable for the health service 
system, that it can only be held accountable for 
providing trained medical personnel, research find- 
ings, and perhaps some areas of social activities 
that contribute to the development of good health 
services. 

Another way of looking at this problem is to 
try to find one, two, or a relatively small number 
of overall indexes concerning the quality and quan- 
tity of the benefits of higher education or attempt- 
ing to find a long list of measurable products of 
higher education. The proponents of measuring the 
immediate products argue that a few gross meas- 
ures may provide some sense of public satisfaction 
but will not get at the long-range problem of im- 
proving the management of our institutions of high- 
er education, nor of explaining to those responsible 
for supporting the institution why a particular 
course of action was taken within an institution. 
While members of the legislature and the public 
they represent may feel it wise to evaluate the 
product of a particular portion of an institution — 
considering its worth with the view of buying more 
or less of it — gross measures will not provide the 
sophistication necessary to make these kinds of 
evaluations. They argue, therefore, that a long list 
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of identifiable products needs to be measured in 
order to provide appropriate analytical tools to 
understand the process of higher education and the 
economics of its funding. On the other hand, we 
are told that “We must be aiming at something 
simple, we must not produce measures which can- 
not be understood by the public, by the government, 
by everybody at the national and local level.*' 

In the Appendix to this publication will be 
found a list of instructional, environmental, re- 
search, and public service outputs of higher edu- 
cation with variables and sources of measurement 
noted. It was developed at a brainstorming session 
after the “Outputs” Seminar had been concluded. 
It is in no sense final and should be considered a 
working draft, a starting point for discussion and 
review. At the same time, a review of this presen- 
tation of the outputs of higher education will in- 
form an interested reader of the direction our 
studies seem to be taking us. 

Finally, there is the position that “ ... no 
amount of analytical studies is going to tell the 
American people how much to spend on higher 
education. That question has to be answered by the 
American people, looking into their hearts and say- 



ing, ‘We know what the outputs really are if we 
would admit it and make a decision.* ’* 

Conclusion 

Educational decision-makers are seeking hon- 
est, viable responses to the issues of public ac- 
countability, flagging financial support, and an 
earlier overreaction to short-term need. They are 
seeking forthright approaches to the polarization 
of opinion about the role of the university in a free 
society ordered by law. 

The urgency of our era exhorts higher educa- 
tion to an examination of purposes, priorities, re- 
sponsibilities, and capabilities. Having made a de- 
cision to be both responsible and responsive to the 
needs of society and the individual, the first step 
toward meeting those obligations is a purposeful 
allocation of resources. There is no clearer reflec- 
tion of the values and purposes of an institution 
than a review of its priorities in allocating re- 
sources. Such purposeful allocation will require a 
careful analysis of the activities, outputs, and ob- 
jectives of higher education. 

Surely we can do no less. 
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Co&mic Considerations 

As we consider concepts and measures (and 
perhaps proxy-measures) for the outputs of higher 
education, we wUI be dealing with social questions 
of almost classical majesty. What is the value of 
basic science and the value of the contribution to 
society of the scientists? What is the value of the 
learned person not only in the sciences but in the 
professions and the arts, both to himself and to 
society? And, even, what is the value of learning 
about values in the setting of formal education? 

These are indeed issues of near-cosmic propor- 
tion. But we will have to skirt some of them, and 
others we will have to work on in the spirit of 
prosaic, detailed analysis rather than that of the 
great social critics and philosophers of education 
in the past: Bacon, Jefferson and Franklin, Cardi- 
nal Newman, Lowell and Eliot, John Dewey; and 
in our own time, John Gardner and Clark Kerr. 

The Bill Is Presented 

In a way, it is ironical that this conference 
takes place in 1970, a century after the great ex- 
pansion of higher education began in America, 
or even that it occurs in 1970 instead of in 1945, 
when the last and biggest increment of this ex- 
pansion began to get under way. But now is the 
time when the bills are arriving and appearing op- 
pressive — to the public decision-makers, to those 
who seek to sustain higher education in the private 
sector, and, not least, to the students! And now 
is the time when a wide variety of new analytical 
and planning techniques are being brought to bear 
in a professional way on the problems of managing 
individual colleges and universities and managing 
higher education as a series of regional systems and 
as a national system. 

All of us have reason to be concerned about the 
costs and resource requirements of higher educa- 
tion. Much of our work, in fact, has to do with ef- 
forts to improve the methods of accounting and in- 
formation assembly which will permit better de- 
scription and analysis of these costs — costs of high- 
er education as a sector of activity in the nation, 
from the point of view of institutional costs and 
the cosling oi particular programs within institu- 
tions and from the point of view of the cost of edu- 
cation as seen by the individual student. But we 
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have bumped hard into the question of output and 
its measurement because, among other things, we 
are seeking now to link the resources used to the 
results achieved — in other words, to link inputs 
with outputs. It turns out that in the long history 
of concern about the processes and activities of 
education, we have achieved only a very imperfect 
grasp of the nature of its results. We are having 
to tackle the problems of output definition and 
measurement under forced draft, for higher educa- 
tion has come to the front of public attention both 
as a major social problem and as a major con- 
tributor to social change and economic develop- 
ment. Higher education is seen as perhaps the most 
significant gateway to individual mobility and place 
in our society. Over the next few decades, it is seen 
as demanding a sharply increasing share of gross 
national product. It is seen as demanding resources 
under conditions which are, at the present time, 
not satisfactory to the providers of these resources 
at the state, the national, or the private level. The 
job we have to do is urgent, important, and con- 
troversial. If we had time, we might do well to sym- 
pathize with ourselves for taking it on. 

The papers and discussions scheduled for this 
conference are intended to cover eight individual 
topics and the linkages between them: four clien- 
teles for output are seen (public, private, faculty, 
and students) and four major functional areas of 
educational activity are to be covered (undergradu- 
ate education, graduate education, research, and 
social involvement). I’m sure that we will benefit 
by 'his configuration of probes into the perplexing 
question of higher education outputs. In effect, 
what we are talking about is the need for opera- 
tional definition and the means for measuring 
the goals or objectives of this system or series of 
systems of social activity. 

It mc-st be said at the beginning that higher 
education will go on, in some fashion, whether or 
not we and others like us succeed in obtaining im- 
proved definition of outputs. After all, it has sur- 
vived for a long time without the kind of effort 
toward developing a rationale that we are under- 
taking now. But I think we can reasonably ask 
ourselves who will be interested in the outcome 
of our efforts to define and measure outputs? The 
clients for our efforts appear to be many, and the 
kinds of answers they may be seeking will not 
necessarily be obtained by a single mode of in- 
quiry, First, there are the national and state public 
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policy makers and resource allocators who are 
concerned with major commitments of public funds 
to the various activities carried on by institutions 
of higher education. Second, there are the trustees, 
presidents, academic decision-makers, generally, 
and faculty and students, in particular, of the 
individual colleges and universities. They have 
urgent demands for information pertinent to the 
decisions they face, and they have fears that the 
solutions to output measurement problems which 
we may propose may profoundly affect their in- 
stitutions. 

Most generally, there are the representatives 
of various public and private clienteles who feci, 
mostly rightly but occasionally wrongly, that higher 
education has an impact on them: employers of the 
trained talent which comes from colleges and uni- 
versities; users of basic and applied research find- 
ings which flow from scholarly activity; participants 
in the culture and the value controversies of society, 
who inevitably link themselves with the culture- 
creating and -sustaining activities of our colleges 
and universities; and citizens at large who, as par- 
ents, taxpayers, and critics, display a kind of fas- 
cinated ambivalence about the importance and 
also the hazard of higher education in these 
tioubled times. 

Social vs. Private Benefits 

Perhaps the broadest questions relating to the 
outputs of higher education revolve around the dis- 
tinctions between the social benefits of higher edu- 
cation and the private benefits accruing to business 
firms and to the individual student and his immedi- 
ate family. Throughout the discussions at this 
conference, it can be anticipated that the problem 
of distinguishing between social payoff or gain and 
private payoff or gain will be troubling. So far as 
instructional outputs are concerned, should we re- 
gard them as having to do only with the gain in 
money income or at least in individual utility for 
the student? Or must we also take into account a 
wide variety of implications for the broader society? 
When an individual scholar or an organization in 
a college or university produces new knowledge or 
creative insight or engages in public service, who 
are the clients for this activity, how do we assess 
the direct benefits to them, do they pay for the 
work that is done, and what are the secondary 
impacts, if any? 

I want to make a few comments about the 
question of social payoff in research and public 
service before I move to some discussion of the 
issue of social versus private benefit to the student. 



We generally have a presumption that a new 
fundamental research result in the sciences, a new 
musical composition, or a feat of historical or criti- 
cal scholarship has cosmopolitan consequences. 
The more fundamental it is, the more cosmopolitan, 
almost by definition. This would seem to argue that 
the public investment in baric research has to be 
made by the broadest jurisdiction available — the 
national government, a philanthropic organization 
operating in the national interest, or even a body 
representative of supra-national concerns. Yet we 
observe that the appropriate jurisdiction for mak- 
ing the decision to invest in the creation of new 
basic knowledge is often unavailable — there is 
not a potent and well-funded world organization 
for the support of basic research in mathematics, to 
use just one example. What has to happen is that 
decision-makers whose sources of support for re- 
search activity are much more localized than the 
incidence of cosmopolitan benefit is, must make 
these investments in new basic knowledge. For a 
large country, much of the cosmopolitan benefit 
can be captured for the nation as a whole, and this 
is part of the rationale for the existence of national 
programs for the support of basic research, as in 
the case most prominently of the National Science 
Foundation. But even this jurisdiction has an inade- 
quate level of funding for the work that basic re- 
searchers may seek to do, and we observe that out 
of their own precious and limited resources, pri- 
vate universities function to support basic work, 
and public institutions, drawing support quite sub- 
stantially from state government, also support a lot 
of basic research. 

My own belief is that this is indeed appro- 
priate, even though the cosmopolitan gains will 
eventually diffuse over the world of civilized men. 
But the rationale that I would offer comes in two 
parts, both of which are difficult to substantiate 
analytically and still more difficult to make 
thoroughly plausible to the man in the street. The 
first element of this rationale is that the rate of 
diffusion of new basic knowledge is not instan- 
taneous, and that a locality or region or individual 
nation takes gains from leadership in basic research 
which are, while temporary, of great importance 
to the vitality and progressiveness of thought and 
of the applications of knowledge in the region or 
nation. The second part of the rationale is that in 
the long run, the very capability to offer serious 
education to young people is dependent on their 
involvement, and that of their teachers, in a com- 
bined research and instructional process in which 
the linkages between the two produce a necessity 
for doing the one in order, effectively, to do the 
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other. As to the rate of diffusion of new knowledge, 
I retain faith in a daring and only partly proved 
hypothesis: That the quality of life and the rate 
of economic and social development in a region 
or nation is deeply connected to its willingness to 
support and honor the creative processes which 
result in new fundamental kncwledge. This may 
not be true, as a matter of priority, for nations 
which are in the early stages of modernization, but 
it seems to me to be clearly true of the United 
States, of the advanced nations of Europe, and of 
Japan. I would even claim that the advanced 
regions within the United States owe much of their 
eminence to the lead which a critical mass of 
scholarly productivity provides. As yet we do not 
have a fully developed theory which will demon- 
strate the differential rates of economic and social 
development that can be achieved under circum- 
stances of varying rates of support to the scholarly 
process, and our dilemma is that until we have 
such a demonstrated theory we must, nevertheless, 
try to put the case for the support of fundamental 
work as a matter of faith, or we must reconcile 
ourselves to a very much more prosaic sub-optimiz- 
ing strategy which would reduce the priority to 
basic research. 

This problem of regional development rates 
needs further work in order to prove or disprove 
the regional hypoth> ris and in order to give a 
basis for placing the locus of resource allocation 
decisions as nearly as possible in tandem with the 
locus of broad social benefit. The regional vs. na- 
tional hypothesis is of special significance in the 
United States because the states have traditionally 
suplied a very large part of the public money for 
higher education — and this, in the face of sub- 
stantial rates of migration of people and ideas. 

Two Nations Supportive of Investment 

The differences between social payoff and Pri- 
vate payoff in the instructional process are of even 
greater conceptual importance, but may be some- 
what easier to deal with. Here again, I tend to be 
a broad constructionist and argue that social bene- 
fits very frequently exceed private ones and that 
educational strategies should be based on this ap- 
proach. At the broadest level of social comment, 
I would simply point out that there are two nations 
of the world which have consistently followed a 
very broad strategy of investment in the educa- 
tion of their people: the United States of America 
and the USSR. They have done so under very dif- 
ferent ideological banners and very different forms 
of government. It seems to me that, as nations, 
they have both been able to capture the great bulk 



of whatever excesses of social over private benefit 
existed because their rates of out-migration of 
persons have been low, relative to their domestic 
populations; second, the accession to eminence; 
and, further, the accumulation of power by these 
two countries in the twentieth century has been 
impressive and even terrifying to other countries. 
I believe that the broad strategy has paid off. 

Another view is that there is no excess of social 
over private benefit. The student as private bene- 
ficiary should therefore pay full cost of his educa- 
tion, except for smoothing of access to loan capi- 
tal and possible provision of subsidy to those of 
low income. It seems to me that had the theories of 
Milton Friedman and other proponents of this al- 
ternative view been controlling in the United States 
for the past century, we would have had a great 
deal less total investment in human capital than 
there has been; and to the extent this investment 
did occur, it would have resulted in a great deal 
of reinforcement, from generation to generation, 
of entrenched economic and social advantage. 

Professor Howard R. Bowen, whose writings 
on the economics of higher education are familiar 
to many of you, has written a paper recently which 
I hope will soon be generally available. It is called, 
“Finance and the Aims of American Higher Edu- 
cation,” and in it Bowen has an appendix, listing 
what he feels are important types of social bene- 
fits (as distinct from private benefits) of higher 
education. Bowen lists an even dozen types of 
social benefits from instruction, and he also points 
to social benefits from the colleges and universities 
as centers of research and scholarship, from their 
presence as pools of versatile talent, and from their 
activities as patrons of the arts. 

It is one thing for us to make these claims of 
the social benefits of instruction, but quite another 
to find good measures for them. If Bowen's list is 
nearly correct about the conceptual elements to 
consider, we will need information about the post- 
graduation activities of former students so that we 
can construct indices of civic participation, cul- 
tural interest, choice of occupations having intrinsic 
social value but not high market income, and other 
social factors. The important issue here is: What 
difference does an exposure to higher education 
make in the life patterns of those who get it? Edu- 
cation is a transformational process, both as to the 
social benefit and the private benefit conferred, 
but I am inclined to think that its transformational 
character may be even greater on the social benefit 
side. 
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Student Progress Viewed 

The student's passage through a particular edu- 
cational program can be viewed in three ways at 
the time he completes it. He meets a standard of 
competence or certification; he may be rated among 
those who pass in relative performance (“number 
one in the class”); and his final performance can 
be gauged in relation to his level of capability at 
the time he began. Let me make a few comments 
about each of these. 

For many purposes, the fact of earning a B.A. 
degree or a Master’s degree may be more important 
to the student making his way in American society 
than the reputation of the institution where he 
earned it, or how well he did relative to his class- 
mates, or what he actually learned. The certifica- 
tion effect seems to have real importance, both to 
the student's self-esteem and to his early oppor- 
tunities for job placement, For an institution, the 
number of earned degrees produced in each type 
and level of program is an important measure of 
its achievement. 

What significance has the relative measure, as 
in the class ranking, or the award of Phi Beta 
Kappa? Based on the internal competition among 
classmates at a given institution or in a given pro- 
gram, it is a competitive signal both to those rated 
high and those rated low. It is also very easy for 
the student, the institution, and the world at large 
to misinterpret. Employers are often very eager to 
obtain information about relative standing, but it 
would perhaps be a more acceptable measure if 
made against a larger population than the college 
class in which the student happened to find him- 
self; in the context of internal, antagonistic com- 
petition the notion of exact relative standing has 
come under attack. The current academic climate 
condemns the more egregious abuses of competi- 
tive, grade-getting behavior. 

The third way of viewing successful completion 
of an academic program is in many ways the most 
sophisticated and the most interesting: What has 
the student attained in relation to his capability at 
the starting point? This concept approximates edu- 
cational value-added. It may be useful within an 
institution, particularly if there is reasonable as- 
surance that tht admission data and the evidence 
of the student’s performance in the institution are 
reliable indicators. But it would be even more useful 
if there are external referents for both — e.g., 
C.E.E.B. achievement scores at the time of fresh- 



man admission and the scores on the Graduate 
Record Examination at the time of graduation 
from college. According to this view, the student 
and the institution together have accomplished 
more education if the value-added is great, rather 
than if it is small. According to this view, an edu- 
cational process which moved a student from the 
lowest quartile of high-school achievement to the 
second quartile of college-graduate achievement 
would be accomplishing something tremendous, 
whereas the college which accepted students only 
from the top decile of high school achievement and 
delivered them into the top decile of college 
achievement would be doing relatively much less. 

As we all know\ status and reputation among 
institutions of higher education are mostly based 
upon absolute standards or, at least, upon the 
horse-back perception of what they are. If we can 
bring greater attention to educational value-added, 
we will accord more credit to the institutions and 
programs where substantial educational gain occurs. 

The number of degrees produced by program 
and level, then, provides one measure of the amount 
of instructional output by an institution or a sys- 
tem; and I have suggested as another measure the 
amount of educational value-added obtained by 
each surviving student. This still does not deal with 
two problems for which current measures could be 
obtained: the amount of attrition, that is, the pro- 
portion of students who do not attain a degree 
and the interpretation of what they gain from edu- 
cational exposure, even though they do not “suc- 
ceed”; and the current measurement of the quality 
of education received. The eventual college drop- 
out has had at least one thing delivered to him 
which our society values greatly: the opportunity 
to try! In fact, we may very well find that as col- 
lege-going opportunity is increasingly universalized, 
attrition rates will increase, and we will tolerate 
this to the extent that the opportunity to try is 
valued in relation to the cost of providing the edu- 
cational exposure. From two other points of view, 
however, we have reason to be very concerned 
about the phenomenon of drop-out. Drop-out 
usually signals failure to the student, and it may 
exact a heavy psychic cost from him. And the 
investment cost of providing the educational ex- 
posure is largely unrequited, we have reason to be- 
lieve, if the student does not persist to the degree. 
More research is needed on this question, but the 
fragmentary evidence that I have seen seems to 
indicate that in terms of market return, at least, 
the drop-out student has a later income pattern that 
is not very different from what it would have been 
if he had not attended college at all. 
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Budgeting Still Bated on Student Flow 

Certainly a major area of efficiency gain avail- 
able to American higher education (as well as to 
elementary and secondary education, where there 
is an immediate social ciisis over the issue) would 
be to increase the achievement and success rates 
of students who presently fail to complete pro- 
grams. Yet educational budgeting in both private 
and public institutions is still very largely based 
on enrollment flow and not on the net output of 
those who achieve well. More knowledge of the 
reasons for drop-out and more attention to the 
design of educational programs to produce success- 
ful achievement would very probably pay big divk 
dends, It is hard to interpret drop-cut by students 
who could succeed in conventional terms, but who 
are repelled by conventional success and “making 
it,” or prefer the Movement, or are in quest of a 
different life. We do not yet know how to confer 
dignity on downward mobility, I might add that 
our universities could very well have produced 
Pontius Pilate and the Scribes and the Pharisees, 
But what would they have done with Jesus of 
Nazareth? 

Quality measures are also very much needed 
now, and they would become still more important 
if new emphasis were placed upon budgeting in ac- 
cordance with the net output of degrees attained. 
(Many academic people express the fear that if 
budgeting were done according to degree output, 
success rates would be improved simply by re- 
ducing the passing standard.) 

There is great cynicism about the educators’ 
alleged preoccupation with educational quality. 
One prominent educational spokesman in Califor- 
nia government put it this way: “Quality, shmali- 
ty!’\ he said. He implied that whenever really em- 
barrassing questions were raised about the amount 
of resources it took to do the educational job, or 
about teaching loads, or a host of other institutional 
practices, educators would defend the status quo by 
arguing that their practices were necessary in order 
to maintain educational quality. I have no sympathy 
for this man’s view or the way he expressed it, but 
I do think that it is very urgent business indeed to 
find measures of the quality as well as the quantity 
of education. 

At the present time, great reliance is placed on 
the status rankings and indicators of institutional 
reputation which are formed in people’s minds 
from very fragmentary folk-lore impressions. Surely 
more can be done. At the crudest level, every 
graduating senior from every accredited institution 



could be expected to take the Graduate Record 
Examination or one of the analogous professional 
school examinations. By analyzing the resulting 
body of data, it would be possible to look both at 
the means and the distributions of each institu- 
tion's graduates in relation to the whole popula- 
tion of graduates. 

Finer measures of quality are undoubtedly 
needed. At some institutions, the administration 
seeks the judgment of outside visiting committees 
in each discipline so that something more will be 
available to them than the special-pleading claims 
of their own departmental faculties. 

Longitudinal data concerning the jobs and ac- 
tivities of former students (both degree- winners 
^apd drop-outs) after they have left an institution 
would Ije very helpful in assessing what the students 
have really gained in educational output. There 
are bound to oe serious arguments about values in 
the interpretation of such information, but it is 
needed both for improved institutional manage- 
ment and in the interest of accountability. Ques- 
tions of tlTe “whole man*’ and of character dropped 
out of focus for a time in favor of attention to the 
graduate's formal academic prowess. But events 
and our national agony should cause us to rein- 
state these issues. Three other value-loaded issues 
are bound to come up. One is the interpretation 
of the value to be placed on the education of wom- 
en who later spend a major part of their adult lives 
in ch ;i d-rearing rather than in the labor force. Is 
the educated woman who does not work simply a 
consumption good for her husband? That was the 
assertion of one economist I talked with, who saw 
no problem in requiring the husband to repay loans 
which had been used to finance his wife’s educa- 
tion. I found this male chauvinism appalling. 

The second issue that is bound to arise con- 
cerns the later migration of educated persons from 
the states where their education had been, in part, 
financed from state tax moneys. What interpreta- 
tion are state authorities going to make of longi- 
tudinal information which discloses that a portion 
of the former student population has migrated else- 
where? Is the state cost of their education an unre- 
quited investment of the state in question? 

Finally, we must point to the deep collision of 
values between those who adopt conventional in- 
terpretations of the worth of education as prepara- 
tion for highly valued occupations and those who 
argue that this is simply supporting a social system 
that needs trained and docile manpower, whereas 
the real point of education should be to prepare 
young people to reshape society. I have no easy 
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answer, but I will forecast that this argument will 
continue for some time. 

Accountability Demands Selectivity 

I want to comment, in conclusion, about the 
importance of output measures for the planning 
and management of the individual college or uni- 
versity. Much of what l have said bears on ways 
to vie»v the individual student as an embodiment 
of educational output, but measures of the per- 
formance of populations and sub-populations of 
students who come out of particular academic 
programs can be interpreted as indicators of insti- 
tutional performance. These indicators need to be 
joined much more firmly than they have been with 
decisions about priorities and resources. 

Each degree program, each academic depart- 
ment, each research institute can be thought of as 
a contributor to the strength (or weakness) of the 
institution. Many of us are working on the slippery 
problems of costing. But costs cannot be gauged 
against too immediate a set of measures of student 
performance (or faculty research performance). It 
usually takes a great deal of time to build up a 
faculty cadre for an effective academic program, 
still more time for that program to achieve reputa- 



tion, and still more time for resources to be with- 
drawn from a program that becomes obsolete or 
useless. What does seem to be important, especially 
in the formation and support of graduate programs, 
is to assess from the beginning what it will take to 
produce a critical mass of faculty and other re- 
sources that will be necessaiy to mount an educa- 
tionally viable and ccononvcally effective program 
and to move toward this target level rapidly — un- 
til it is reached, costs are very high in relation to 
results. From the institutional standpoint, critical 
mass is also important because it provides a means 
of delivering institutionally-recognizable outputs, 
rather than outputs (in teaching or in research) 
which are attributable solely to individual students 
or faculty. 1 see no evidence that the decade of the 
1970s will be very different for American higher 
education than the first year of the decade has 
been. If this is a valid presumption, the require- 
ments for efficiency improvement and for coura- 
geous selectivity in the allocation of resources to 
competing programs will lay heavy responsibilities 
upon college and university administrators and 
faculty. Both external accountability and the neces- 
sities for coherent internal priorities will force a 
rapid increase of attention to the outputs of every 
institution. 
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"I represent the ” foreign " element In your meeting ." 

“It It Interesting (hat the size of the public socfr r In most of 
the advanced Industrie I democrecles It strikingly similar. 
Whether you examine an extreme social democracy like 
Sweden, a moderate social democracy like the United King- 
dom, or an advanced capitalist country like the United States, 
the public sector seems to be roughly the same site. Further- 
more, In all these countries, as Indeed throughout the world, 
the hulk of education expenditure falls firmly within the 
public sector" 

“Money Incomes, however discounted, are not acceptable as 
proxies of educational output. Ob/ectlvo tests of achieve- 
ment are acceptable, provided that they are appropriately 
weighted lor student Inputs." 
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The Outputs of Higher Education — 
Their Proxies, Measurement, and Evaluation 

JOHN VA1ZEY 



First, may 1 say liow honored I am to have 
received this invitation to come and address your 
meeting. The topic is one in which 1 have long 
been interested, and I hope that this meeting, to- 
gether with others which are scheduled on the 
same topic, will enable us to make sonic significant 
advances in research in this area. My wife and 
children are American citizens, and 1 am constantly 
on the brink of becoming one. For me. therefore, 
though 1 represent the "foreign" element in your 
meeting, to speak in Washington, and to meet so 
many friends, new and old. is like coming home. 
I am doubly grateful, therefore, for your invitation. 

My own work has been concerned with ques- 
tions of measuring input and productivity, and 
therefore 1 have paid particular attention to the 
problem of output.* What I have hitherto done is 
to take for granted the question of output: to as- 
sume. that is. that if there arc two alternative meth- 
ods of achieving the same objective, the objective 
would be equally well or badly attained by either 
route. This assumption of constant output enables 
comparisons of inputs nrd comparisons of tech- 
niques to be made easily, since the mathematical 
formula which is required is obviously much sim- 
pler, and because economists then have no need 
to involve themselves in questions of measurement 
of output which raise Issues of psychological and 
other forms of evaluation that are much disputed. 

The Market Mechanism 

The market mechanism, as (aught in neo-classi- 
cal economics, solves most of these problems of 
output measurement, subject to the acceptance of 
one enormous simplifying assumption. In the neo- 
classical economic system, money prices, both of 
input factors and of intermediate and final goods 
sold on the market, frpresent scarcities, and the 
crucial assumption is made that there is a constant 
marginal utility of mone/ so that scarcities in one 
area can be compared with scarcities in every other 
area. It follows, therefore, that if a teacher costs 
$20,000 a year and teaches 20 children, the cost 

»ork to which I refer T * le Economics of Fauot i* 
tionaf Costing. in four votufrx<, of whkh the first three 
have either jwi appeared or will }omtl) do so. available 
from the Centro de tceivcmia e Financas. Gulbenkian 
Foundation. 56 Avenida d« Berna. lisbon. 1., Portugal. 



of that education is equal to SI. 000 per year per 
child, and in a perfectly competitive market system 
the education would be sold at that price to those 
children and would — end here wc come to a sec- 
ond crucial asumption — it would, for the mar- 
ginal child, be worth exactly the benefit that the 
child derives from it. 

Now there are many merits of the neo-classical 
system, particularly in its Austrian formulation, 
which has become so dominant a mode of eco- 
nomic reasoning in the United States. Chief among 
them is algebraic elegance and simplicity of policy 
prescription. If a paradigm of the market can be 
set up we have no need for proxies. Output is 
measured by the value that it is sold for. and in- 
puts cost exactly what is paid for them. It follows 
that in a free and perfect market, the price of a 
year at school — say $5,000 — would be “worth M 
$5,000 to the marginal family, and that is all there 
is to it. The difficulty with this type of analysis is 
that, firs 1 of all. it rests upon assumptions which 
may well be questioned, such as whether or not 
there arc perfect markets for factors and perfect 
markets for intermediate and final products, and 
secondly it avoids all questions of economies of 
scale, of monopsony, and of monopoly, But. third- 
ly. and above all. it leaves on one side the ques- 
tion of the allocation of resources within that very 
large part of economic and social life which is now 
the responsibility of public bodies. 

Parenthetically, I may say that it is interesting 
that the size of the public sector in most of the 
advanced industrial democracies is strikingly simi- 
lar. Whether you examine an extreme social de- 
mocracy like Sweden, a moderate social democ- 
racy like the United Kingdom, or an advanced cap- 
italist country like the United States, the puWk 
sector seems to be roughly the same size. Further- 
more. in all these countries, as indeed throughout 
the world, the bulk of education expenditure falls 
firmly within the puWk sector (excepting only the 
somewhat controversial Instance of income fore- 
gone by students while they are studying). This 
leads me to assume that the problems of measure- 
ment of input, productivity, and output in the 
public sector assume a similar importance for many 
countries despite differing ideologies — economi- 
cally, sotia.ly. and politically. 1 take it. therefore, 
that the question that we are debating is one where 
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international experience is strikingly relevant and 
important. The case is different for a service that 
in one countiy is in the public sector and is out 
of it in another country. The National Health 
Service is an example here. 

You will be aware, and it will doubtless be 
frequently mentioned in this Conference, that the 
technique which has become prominent for cal- 
culating the relationships of output to input in the 
public sector goes under the name of “benefit-cost 
analysis” or — something similar, “cost effective 
techniques/* These techniques date, iu essence, 
from Pigou*s work on the Economics of Welfare. 
in which he drew attention to the indirect as well 
as to the direct consequences of economic activity, 
and suggested that these effects, which did not 
appear in the sector of society where money was 
the measure, could be estimated by analogy with 
money costs. Thus, in the field of water conserva- 
tion and in the field of oublic transport, calcula- 
tions have been made of the costs of substantial 
public installations and of the benefits that accrue, 
both directly to the consumers of the products by 
these installations and of the other indirect conse- 
quences. You will be familiar with the work of the 
calculation of the rise in income of a locality which 
develops because of improved transport facilities 
the reduction in pollution which improved public 
transport services provide if private transport de- 
clines as a consequence, and so on. Mr. McNamara 
and others are responsible fv extending this kind 
of calculation to defense, though I think it has be- 
come increasingly doubtful as to whether or not 
the calculations made in that field were either 
analytically satisfactory or in the end operationally 
viable. 

Colt Efftctivo RtlalloniMpt Crucial 

Increasing!', there has been a movement for 
other public services to come within the orbit of 
this type of analysis: indeed, with the sp r ead of 
target budgeting, both in Canada and the United 
Stales, in which the objective is stated and alterna- 
tive means of achieving the end are set out, the 
cost effective relationships become crucial deter- 
minants of public policy. It is. therefore, not sur- 
prising that the attempt has been made to bring 
these techniques into the field of education. Sur- 
prise is even less. I think, when regard is paid to 
the growing proportion of the national income 
which is demoted to education, which is now ap- 
proaching 7 per cent in many countries, and to the 
crucial role that education plays in the provision of 
skills and in the determination of social mobilil) 
and other social characteristics of the population. 



One point should be carefully noted. Benefit- 
cost analysis provides but small opportunity for 
contrasting different sectors of the economy; it is 
only applicable within sectors, and is there to com- 
pare and contrast activities which closely resemble 
each other. And another point that will become 
critical in the analysis that follows may often be 
forgotten. The valuation that is put upon the non- 
monetary benefits and losses is subject to dispute, 
as it has not been settled by the market. So. we 
can never escape an index-number problem. 

I. therefore, wish to examine in some detail 
the question of evaluation of output, First. I think, 
we have to realize that \vc are concerned basically 
with three sorts of questions. The first is what can 
be defined as output? We have to remember that 
one of the outputs, namely the student who emerges 
from a system, is at the same time an input and 
also part of the productive process. If we there- 
fore take the formula 7T (Si) S n we find that 
there is a common element in all (he symbols 
which are being manipulated. As I have argued, 
this is a matter which is fundamental and deserves, 
I think, very important attention, because it docs 
affect profoundly the arithmetic, as well as the 
policy prescriptions, which might arise from the 
arithmetic. 5 think we have to consider very deeply 
the meaning of the words input, output, and the 
ratio between input and output which is variously 
labeled, “cost effectiveness.” “value-added.” or 
“productivity.” For example, the input to a given 
year in college is the student's experience of the 
work of the previous year. One of the benefits or 
consequences that atises from the input Is the 
actual experience or enjoyment of the year in 
itself. There is then the output, perhaps more con- 
ventionally measured, of the year which could be 
defined in terms of examination grades, or in other 
ways. It is very important to understand that these 
three separate stages art all closely linked with 
each other and. in some sense, indissolubly linked, 
so that to attempt to divide them out is to attempt 
to dissolve the indissoluble. I shall revert to this 
point later on in my speech. 

I want to emphasize that far from knowing 
what is ft and what is S* and S r . we could re- 
phrase the formula in the form of ? (?) ft ? which 
would give us a far more realistic description, in 
my view, of the process of education as it is carried 
on than some of the calculations that I and others 
have made. 

It is extremely difficult to measure inputs, 
even when such relatively non-controversial meth- 
ods as estimating the true costs of teaching are in- 




21 



volved. I want to call attention to this matter in 
relationship to international comparisons of educa- 
tional outlay. Particularly, I want to draw atten- 
tion to (he question of evaluating student inputs. In 
the first case, you can simply take numbers of stu- 
dents, but since we know that the ability of differ- 
ent students to study is related to other socio-eco- 
nomic variables, the inputs of studems have to be 
weighted by such matters s sex, race, socio-eco- 
nomic background, and ability. Ideally, as we ali 
know, for comparing the progress of two students 
they should be similar in every respect. The ex- 
treme unreality of the assumptions becomes clear 
in any basis of comparison, since measuring the 
ability or previous educational experience, or even 
the socio-economic background of different stu- 
dents is not, ever, an unambiguous or uncompli- 
cated procedure. 

Nevertheless. I do not want to be too depress- 
ing. Techniques have been evolved fot doing this 
which are perfectly well-known; and by an analysis 
of variance I think that substantial attempts ai 
standardizing student er.'ry can be made. Further- 
more. for any one inslTition differences from one 
year to another are not likely to be substantial. 
Nevertheless, I want to make quite clear at this 
stage that already when we evaluate student inputs, 
we have an index number on our hands in the 
sense that we have weighted the input for different 
characteristics and the question of which weights 
we adopt will affect the outcomes. 

Teaching Procedure* Studied 

The next stage is to vary the procedures by 
which students are Interested, and I would draw 
your attention to the fact that Profcs;or Tor- 
sten Husln has already said that so little is known 
about the different procedures of teaching that to 
contrast, say, teaching by television with “conven- 
tional"’ teachirg procedures is itself an extremely 
summary procedure. In addition. I think that Larry 
Segal’s work on the study of the influence of tele- 
vision has drawn attention to the fact that even 
in an area which would seem to be as relatively un- 
ambiguous as the effect of television instruction, 
the work divides pretty evenly into those who say 
that its effects have been beneficial and those who 
say it has not. As I recall it. this was a study of 
over a thousand papers and. therefore, his results 
are not unimportant, especially when they are con- 
trasted with what is known of the difficulty of ob- 
serving the classroom situation. 

I want to say a little about observation of the 
classroom situation. Obviously, the ideal procedure 



for describing what goes on in the classroom is 
some attempt at participant observation. Of course, 
in universities dealing with siudents of university 
age we have ideal opportunities for this form of 
development; and there are well-known psycho- 
logical procedures for calibrating and evaluating 
the different methods achieved by participant ob- 
servation. I will not expound them here. I merely 
draw your attention to them and pass on. 

1 turn now, therefore, to the major question 
which is the theme of this Conference: outputs — 
(heir proxies, their measurements, and their evalu- 
ations. I have taken this long way ’round to reach 
the point because I wanted to emphasize that evalu- 
ation of the outputs is not independent of the 
evaluation of the inputs or of (he procedures by 
which you reach the outputs and that, therefore, 
many of the techniques which are used for meas- 
uring the results of industrial or economic activity 
are not necessarily applicable to education. It may 
well be that the devices which we should use for 
this sort of judgment should be derived from the 
discipline of social anthropology or social psychia- 
try. This seems to be a realm which we should dis- 
cuss fully. 

Now the first and most important method of 
analyzing outputs is by simple numbers, by which 
you relate the number of students who go into a 
system to the number of siudents who emerge from 
a system. Until recently, for international compari- 
sons. this was virtually the only method that we 
had and simple though it may seem, the process of 
achieving statistical conformity for comparing dif- 
ferent nations is stil’ in its infancy. Despite this, the 
crude comparisons between such things as the pro- 
portion of the age group that go into higher educa- 
tion in different countries and the proportion of 
the age group that drop out in different countries 
seem to nie to be important and significant indi- 
cators when there are gross differences. Another 
measure which seems to me to be important in 
the school system is the number of repeaters; in 
many countries it is a common habit for children 
to be forced to repeat a year and this ha$ a cumu- 
lative and deteriorating effect on educational out- 
comes. 

The next series of procedures are tests and 
grades and examination scores of all kinds. Here 
we arc faced with a number of complex problems. 
The first is the extent to which the examination re- 
sults ere consistent; evidence is widely known for 
many years that in the essay type of examination 
or the oral examination, which is common in Con- 
tinental Furope. results given by different examiners 
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are highly inconsistent. Even the same paper 
graded tvs ice by the same examiner will be given 
widely different grades. It may well, therefore, 
seem that objective testing is more important. But 
objective testing raises difficulties when it is used 
infcrlcmporally, that is from one year to another, 
because of the way in which knowledge changes, 
and also interculturally in the massive educational 
achievements study which has its headquarters in 
Sweden, perhaps one of the most significant pieces 
of social research now being done in the world. 
It has been found difficult even to compare mathe- 
matical outcomes in different countries, and once 
such subjects as civics and civic education are 
raised, the problems become extremely complex. 
To give you an instance of <hc problem, may I 
put it tike this? Supposing I asked children in 
twenty different countries, “Who is the father of 
your country?" The United States child might 
answer George Washington; the child in Russia 
might answer Stalin; the child in China might 
answer Mao Tsc-tung. Now if wc posed a ques- 
tion about Stalin, Mao Tsc-tung. and George Wash- 
ington in all three countries, the children would 
perform differently. Therefore, wc have to find — 
if we are making a comparison between these 
countries— something which is culturally neutral 
with respect to all of them. In the Western world 
perhaps we might choose the Beatles, or some 
other group. But this, of course, carries with it the 
value judgment that what is special or particular 
to a particular culture Is less important than what 
is common to all the cultures which are being as- 
sessed. This assumption is itself a value judgment 
which is not acccp'aWe. 

We then have to relate these tests and achieve- 
ments to the inputs which arc relevant. To take a 
particular example, in the mathematical study it 
was found that the Japanese performed particulitly 
well and the Americans and Swedes performed par- 
ticularly badly. The Japanese concentrated on a 
small percentage of the age group, whereas the 
Americans and the Swedes were undertaking the 
heroic process of attempting to educate a whole age 
cohort. It necessarily follow*. 1 think, that as you 
go down the socio economic scale and as you go 
down the attainment and ability scale which, as we 
know from socio-eeonomk research, are much the 
same thing, the level of performance which can be 
expected will drop off. How do you weight the 
levels of performance by different strata of the age 
cohort? Again there are techniques for doing so. 
but 1 have to draw attention to the fact that the 
weighting we adopt wifi affect the output. 



Welshing lor Failure 

We then come on to the question of failure. 
Supposing that for anyone taking a given test, 50 
per cent succeed and 50 per cent fail. Do we 
weight for the 50 per cent who succeed a r > having 
attained 100 per cent of the objective and the 
others zero, or do we grade them according to the 
proportion of the ultimate objective that is 
achieved? One of the most common problems in 
examining is whether or not wc are setting a level 
of attainment which all should achieve. The analogy 
which 1 draw here is with the examination which 
enables you to drive an automobile — an examina- 
tion which, ultimately, almost all people who are 
physically normal ought to be able to pass. Or are 
wc dealing with an examination which is designed 
to select only very few people of extremely high 
and rare talent, for example, the kind of test which 
might admit a potential ballerina into the Bolshoi 
Ballet? 1 am not suggesting that there is any one 
answer to this; obvicusty, there is not. What I am 
suggesting is that the kind of weighting that we 
adopt will affect the outcome ancf that this kind 
of weighting needs particular attention. 

The next range of objectives for which tech- 
niques of evaluation exist a'c changes in person- 
ality and attitudes. Here aguin. there are consider- 
able difficulties, both in testing before admission 
and testing after the conclusion of the course We 
have problems uot only of weighting changes in 
attitudes against the overt desire to change those 
attitudes, but also the problem of weighting char- 
acter and altitudinal changes sgTnsl academic 
performances. There are hosts of other values 
which are important and relevant — the general 
level of culture, sporting achievements, a whole 
range of activities which are assumed to bo the aim 
of activities in universities and colleges. Bloom's 
Taxonomy of Educational Objectives is extremely 
helpful here, but it gives no quide to the relative 
weighting that should be adopted. 1 think *he only 
answer that one can suggest for this relative weight- 
ing is the one of pragmatism, that what is inevitable 
and necessary is for the agreed objective formula- 
tion to be presented, much in the way in which a 
Bcrgsonian welfare function wx>uld be formulated. 

W (U 1 tH). 

Income ftalatttf to Attainment 

This brings me to the last but not the least im- 
portant section of the theme that I w ant to develop 
before you. This is the evaluation of education out- 
put of the consequences of what happens to the stu- 
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dents. T’nc best formulated and best known of these 
is, of course, the relationship between educational 
attainment and income. The procedure will be fa- 
miliar to you, that the lifetime earnings profile of 
people of different educational attainment is re- 
lated to (hose educational attainments, after stan- 
dardization for other characteristics, which socio- 
economic background raises, and ability. I have 
argued elsewhere that this process of standardiza- 
tion is by no means as obvious a statistical pro- 
cedure as it may seem because the variables in the 
relationship are very closely inter- related. 1 have 
been supported by Professor Herman Wold of the 
University of Uppsala, and Professor Torsten 
Hus£n of the University of Stockholm, In my 
suspicion that the statistical problems hero are 
much larger than has been assumed in some recent 
work, and I would refer you for a full discussion 
of this theory to a paper which has been published 
elsewhere. 

But this relationship between education and in- 
come. even if the problem of standardization is 
overcome, then presents formidable problems. For 
one thing, there is considerable statistical difficulty 
in extending cross section data into a time series. 
Secondly, the hypothesis upon which judgment is 
made, namely, that income differentials will remain 
the same, is open to serious doubt. Thirdly, the as- 
sumption that tne labor market is functioning per- 
fectly is even more questionable. Despite these dif- 
ficulties, it might be argued that this kind of calcu- 
lation is mosi useful and most important. 

I have real doubts about this as a test of edu- 
cational output; nevertheless, other people do sup- 
port these particular techniques. 

The question that arises now. t think. Is the 
relationship which should be established between 
all erf the outcomes of which I have sooken. In 
addition the question of pleasure and satisfaction, 
or erf displeasure and dissatisfaction, w hich students 
themselves experience during their lime at the uni- 



versity must be considered, for this in itself is one 
of the outcomes of the education process. I can 
only suggest as a technique the iteration of costs, 
techniques, and consequences in as full detail as 
possible, in order that they may be all weighed up 
together, according to whatever social welfare func- 
tion seems suitable for the purpose. 

That, however, seems too negative a line at this 
stage. May 1 perhaps make some concrete proposi- 
tions to be accepted or rejected? 

1. Money incomes, however discounted, are 
not acceptable as. proxies of educational 
output. 

2. Objective tests of achievement are accept- 
able, provided that they are appropriately 
weighted for student inputs. 

3. Other charac;eris(ics that can be tested 
should be added to these objective tests of 
achievement by formulae that must be ex- 
plicit and must be agreed upon. 

Now. as to techniques, my one prejudice is very 
strongly in favor of longitudinal cohort studies. I 
realize that (his is much more expensive than other 
modes of evaluation, but I believe (hat evidence — 
whether Hus£n’s. the International Educational 
Achievement Tests, and Dr. Douglas's studies — 
all supports the notion that longitudinal studies are 
the best. It is my own conviction that other tests 
which are not part of a cumulative scries tend to 
exaggerate the importance of one factor as against 
another. 

Monitoring N«e«t«ary 

This raises the difficulty, an obvious one, of 
how to do it. The more I study this question the 
more convinced I am that in my country — and 
presumably in other countries, too — we should 
have a continuous monitoring process for evaluat- 
ing the results of education. 
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"T/me/y It it for educators fo define obf actives, precltely... 
to develop measurement devices . . . and to offer a quantita- 
tive model for fudging the success or failure, the adequacy 
or Inadequacy, of higher education In attaining desired 
goals." 

"The challenge It to Identity measures that meet all criteria, 
tor at a rule consensus measures are not quantitative and 
additive, and quantifiable goals are not generally accepted. 
Pragmatically, achieving all characteristics 100 per cent It 
Impossible. Compromise It essential." 

" The growth that occurs within this collegiate environment 
may be, for the convenience of goal setting and measuring, 
grouped Into five categories: Whole man growth . . . special- 
ised man growth ... growth In the pool of knowledge... 
growth In society-at-large ...the foy of growing and being In 
an educational environment." 
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A Scheme For Measuring the Output of Higher Education 

DAVID G. BROWN 



Too long higher education has trusted (he 
canard that “ . . . college is good for you and 
benefits the nation 1 ’ and has neither defined ob- 
jectives nor measured response. Now donors and 
legislators are demanding proof. Now students 
question the wisdom of college for all. Now edu- 
cators arc presented with a plethora of teaching 
methods, subject matte* s. and educational en- 
vironments without the tools to discriminate be- 
tween the effective and the inefficient. 

Timely it is for educators to define objectives 
precisely (Section I of this paper), to develop 
measurement devices (Section 2), and to offer a 
quantitative model for judging the success or fail- 
ure. the adequacy or inadequacy, of higher edu- 
cation in attaining desired goals (Section 3). 

How easy it would be to measure output as the 
number of blue-eyed students graduated, the num- 
ber of words published on pink paper, and the 
number of times Playboy magazine was checked 
out of the library. These nonsense measures do not, 
however, value outputs or measure output in terms 
of objectives. To construct meaningful measures, 
statements of objectives must precede. 

The Objectives 

Universities provide multiple products. Funda- 
mentally, the university is a growth environment. 
Here, resources are brought together for the con- 
venience of the student who wishes to grow. Here, 
learning is n>orc efficient because of the proximity, 
the extent, and the diversity of resources. Here, 
also, the scholar, as he strives to extend the bound- 
aries of knowledge, is supported by the environ- 
ment. 

The growth that occurs within this collegiate 
environment may be. for the convenience of goal 
setting and measuring, grouped into five categories. 

‘Originally I outlined an ankle that might appear in the 
Amtrkun Economic Fragments of that *'*rtkle~ 

Mill remain fn the main, however, this paper is written 
to prevoke discimion and to highlight ksucs rather than 
to he precise and fair in a scholar?)* manner. Rather than 
huikling from pan hieratnre f have chosen ihe more 
controversial technique of jumping beyond my data and 
describing a model that is still in its infancy. Hopefully 
in morling hack to hard fact we shall have imighls ett 
output in hieher education. In writing this paper I am 
esoecially indebted to Dr. Curtis Page, Assistant Provost, 
Drake University, for his wisdom and e occur agement. 



(1) Whole Man Growth — developing or 
reaching toward a value system, a familiarity with 
the heritage and the culture, an understanding of 
self, an ability with inter-personal relations, a life 
style and life goal. This growth may be achieved 
by many means, from experimentation with differ- 
ent life styles to a library-based study of life styles 
in various cultures. Normally this type of growth 
is the emphasis of undergraduate education. Com- 
ponent objectives include increasing the student’s 
ability to: 

A. Lcam to feel (e.g., compassion, love, con- 
cern) 

B. Learn to retain facts 

C. Learn to think (i. c.» logic, methods of an- 
alysis) 

D. Learn to decide (l.c., philosophy of life, val- 
ue system, methods of analysis) 

E. Learn to act (e g., do. create, communicate) 

F. Learn to learn 

(2) Specialized Man Growth — developing 
motivation and skill as a specialist, an in-depth un- 
derstanding of one cvea of knowledge, a narrow 
expertise. This may be achieved through appren- 
ticeship or through study of methods and styles of 
past masters. Normally this type of growth is the 
emphasis of graduate and professional education. 
Component objectives include increasing the stu- 
dent's ability to: 

G. Choose a career 

H. Gain admission to next stage In career de- 
velopment ( \ g., medical school) 

I. Develop skills needed to fulfill career 

J. Earn a Jiving for self and family 

K Fulfill society's manpower needs (including 
discovery of talent) 

(3) Growth in the " Pool of Knowledge" — dis- 
covering new* relationships, new phenomena, new 
materials for the purpose of stockpiling knowledge 
and broadening the base for further discos ery. 
These discoveries may be in the research labora- 
tory or off the campus, and by either students of 
faculty. This group of discoveries differs from cate- 
gories one a. id two where the discoveries are more 
personal, the discoveries result from a new person 
entering territory previously mapped. Normally 
this type of growlh is the emphasis upon research. 
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Component objectives include ability to: 

L. Identify new phenomena 

M. Synthesize and summarize in new ways 

N. Communicate new knowledge to others 

(4) Growth in Society-abLarge.. — applying 
knowledge to societal problems, relating discoveries 
in the laboratory to the market, reflecting on the 
value system of society-at-large. These applications, 
which advance societal welfare are providing: an 
educated c:i!zenry f a trained corps of social agems, 
and a basic pool of knowledge. And the contribu- 
tions are more direct. Here the university depots, 
from its primary role as “capital builder” and pro- 
vides a “current good” to society. Normally this 
type of growth is the emphasis upon social involve- 
ment and service. Component objectives include 
ability to: 

O. Create design for new society (e. g., design 
model cities program) 

P. Carry out design (i. e., provide manpower 
actually to do ihc job) 

0. Evaluate sociel>'s current attack on prob- 
lems 

(5) The Joy o) Growing and oj Being in an 
Educational Environment — providing income and a 
way of life to university employees, a joy-giving ex- 
perience to students while in college, and a tinge 
of the intellectual environment to the city or town. 
This output is expressed by students as: “It's a 
hell of a good way to spend four years and a darn 
sight better than the army”; by faculty as: “f cher- 
ish the life of the mind and wouldn't consider 
working for industry”; and by community resi- 
dents as: “1 like living here because the university 
provides easy access to evening courses, dresses up 
the physical appearance of the town, draws many 
interesting people, and brings customer and tax 
dollars to town.” 

Component objectives include ability to: 

R. Provide dignity, self-esteem, and material 
goods to the faculty and others on the pay- 
roll 

S. Provide enjoyment and happiness to stu- 
dents and faculty involved in the educa- 
tional process 

T. Provide benefits. both psychic and real, to 
citizens of the community. 

At the outset it is important to under- 
stand that Products 1*3 differ substantially from 
Products 4 and 5. Products 1*3 may be measured 
as growth or as the change in the value of a capi- 
tal asset. Here, universities are pioducing goods 
that *id, in turn, themselves produce other goods. 



Here, the economic theories most applicable are 
capital theories. 

Products 4 and 5 arc consumer goods, here- 
today-gone-tomorrow. Measurement is not of 
growth but of the joy and satisfaction generated at 
the moment of consumption. Here, higher educa- 
tion's contribution is measured in terms of the 
value today of, say. uncovering the radiation dan- 
gers of nuclear reactors or of restructuring income 
tax tables. It is measured in the psychic and real 
income received today by faculty and students. 
Consumption not growth is involved. Assessment 
of growth, with the requisite measures at the be- 
ginning and at the end. is not necessary. An analogy 
may help. To measure the growth in value of a 
manufacturing plant between 1965 and 1970, it is 
necessary to know a price at both points. This is 
the type of measure needed for Products I through 
3. To measure the value of attending an art exhibit, 
it is necessary only to know the price of admission. 
Attendance at an art exhibition is a consumer good 
that is used up entirely at the moment, and that 
is the nature of Products 4 and 5. The values of 
Products 4 and 5 may be measured only at the 
point of output. The values of Products 1-3 must 
be measured as differences between values rt input 
and values at output. 

Measuring Objective and Output 

Output choice and measurement choice relate 
closely. Unfortunately, broad consensus goals are 
Immeasurable, and measurable goals lack general 
endorsement. The dilemma is arrogance versus 
Imprecision. Avoiding catalog rhetoric and the lofty 
phraseology of committee reports, this quest is for 
an operational measure even more than a consensus 
goal. The technique will be to provide alternative 
measures for each consensus goal, thereby allow- 
ing the model user to employ those measures that 
are “operational for him* (I. c., he has the data) 
and “agreeable to him.” 

This measure must have these characteristics: 

(1) Quantifiabilily. Output must be stated in 
objective, quantifiable units. 

(2) Additivity. The output in Category w 1” 
(i. e., whole man) and the output in Cate- 
gories “2-5" must be stated in the same 
units so that total output may be summed. 

(3) Divisibility. The measures selected for the 
system as a whole (American Higher Ed- 
ucation) must be equally available and 
meaningful for subsystems such as private 
versus puNk higher education, higher 




29 



education in the state, XYZ University, 
the College of Law at XYZ, and a particu- 
lar activity within the XYZ Law School. 

(4) Transferability. The measurement system 
appropriate for 4-year colleges musi also 
be appropriate for universities and for 
junior colleges. 

(5) Consensus Acceptability. The measures 
should be credible. A majority of all con- 
stituencies — students, faculty, public, pri- 
vate — should agree that the measures 
are not inappropriate. 

(6) Flexibility. To the extent that consensus 
acceptability of goals and measures can- 
not be achieved, model users must be pro- 
vided with "mix and match" output meas- 
ures. A slate planner, for example, may 
wish to measure change in tax revenues 
due to higher education and would, there- 
fore, prefer “change in lifetime income” 
to measure output. A scholar may, on the 
other hand, wish to measure disciplinism 
or growth of the mind and, therefore, 
prefer "change in percentage intending to 
attend graduate school” or "change In 
10." The point is that the model must be 
flexible so that users with different pur- 
poses and with different values can em- 
ploy It for their own purposes. Each goal 
must be measurable by several means. 

The challenge Is to identify measures that meet 
all criteria, for as a rule consensus measures are 
not quantitative and additive, and quantifiable 
goals are not generally accepted. Pragmatically, 
achieving all characteristics 100 per cent h im- 
possible. Compromise is essential. 

The realistic choice is between "no measures” 
(subjective judgments) and "imperfect proxies.” 
Criticism that the measures used are not perfect 
isn't relevant. If the proxies operationally assist re- 
source allocation, if decislon-maVing is better with 
the proxies than without, this alone is justification 
for their adoption. Not to use a proxy because it 
reflects only imperfectly a desired output is to wait 
stagnantly for El Do. ado. 

Proxy measures may be grouped as "input." 
"output," or "value added.” One estimate of the 
quality of an education is the number of courses 
offered, the education of the teachers, th.* breadth 
of course offerings, the dollars spent per student, 
and the number of volumes in the college's library. 
The rationale for these input proxies is the very 
weak assumption that availability and exposure 
automatically result in teaming. Output estimates 
(e. g.. number of earned credits. Graduate Record 



Examination Scores, percentage going to graduate 
school, average starting salary of graduates) rest 
upon a similarly weak assumption that all growth 
measured at the end of college is the result of col- 
lege or, alternately, that all students enter college 
with (he same background and same potential to 
learn. 

By far the most meaningful proxies measure 
changes or value added during the college years. 
For sample, if a student reeds 230 w’ords per 
minute when entering college and 345 when leav- 
ing, that the rate increased by 1 1 5 words or 50 per 
cent tells more about the effect of education than 
either the 230 start figure or the 345 finish. It 
might be, for example, that some students would 
start at 400 and drop to 345. Value-added meas- 
ures will not always be available but they should 
be used where possible. The greatest problem In 
developing an omnibus measure of educational out- 
put is additivity. Apples and oranges don't add, not 
do "increases in income potential" and "changes in 
percentage of students going on to graduate 
school." To meet the special need for additivity, 
the Index Number is proposed as the basic measure- 
ment concept. 

The Consumer Price Index is an artificial 
measure of price level change, a measure based 
upon the rather arbitrary assignment of weights to 
bought goods. Similarly artificial are the Dow-Jones 
Market Index and job-rating scales. 

The time is ripe to unleash index makers on 
higher education. Some courageous fool whose fu- 
ture doesn't rest upon his reputation as an Index 
maker must go first, so here goes. 

Assume that — 

M| =s the “ith” measure of output 



V, = the weight assigned to the “ith” measure 
M*- value of the “ith” measure at time "a." 

w, = 1,000 (that is, the sum of the weights 
equals 1,000) 
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A simplistic example, with hypothetical measurement values, will clarify the 
workings of the model. 





May 


May 




MEASURES 


1969 


1970 


WEIGHTS 


Number of degrees granted 


7,000 

(MD 


7,200 

CM?) 


300 

(W.) 


Starting salaries of graduates 


$10,000 

(MS) 


$9,000 

(M, b ) 


100 

(W,) 


Per cent of students rating education 


76.3% 


80.0% 


600 


as good/c.':celIent 


CMS) 


(M?) 


(W,; 



O* = 300 



7,200 

7,200 



+ 100 



10,000 

10,000 



+ 600 



763 

763 



= 1,000 



7,200 

O b = 300 + 100 

7,200 



9.000 , ^ 800 

■ + 600 

10.000 763 



« ? 



O h = 300 x 1.029 + 100 x .90 + 600 x 1.049 = 7 
O* = 308.7 + 90.0 + 629.4 - 1028.1 



1028.1 means that output is 2.81 per cent 
higher in 1970 than in 1969. Here, three meas- 
ures were used; the number might as well be 30 
or 300. Here a 12-month interval is assumed; the 
interval might be five years, six months, or one 
week. Here, the values imply one university. With 
corresponding magnitudes the unit might be the 
state, the region, the nation, or a college, a depart- 
ment. a course, or a portion of a course. 2.81 per 
cent is all it means. Of course, 1028 cannot be 
priced in dollars (unless someone even more a fool 
than I should pr 5 ?e 1.000). Thus our measures 
proside a yardstick for chc.ngt in output only. 

Again, job rating sy.-'.ems are analogous. Here 
jobs are assigned point values that make sense only 
when related to ont another. The dollar valuing 
of points is an independent process. 

Admittedly, a dollar figure for output would 
be superior to a measure of percentage change. 
But a percentage change measure is a gigantic step 
forward, a step that has value by itself and a step 
toward the ultimate goal of a benefit measure that 
is in the same units as costs. 



First Attempt 

So. realizing the limitations, let's proceed to 
a more complicated and plausible model. To iden- 
tify appropriate measures we ask, "What is the 
least imperfect quantitative measure of Objective 
#1? Of Objective #2?'' Answers are provided 
in Table I. 

Throughout Table I appears “student testi- 
mony" and “faculty testimony." By this is meant 
direct feedback by students on whether they think 
they're learning and by faculty on whether they 
think they're stimulating, producing, and/or en- 
joying. Trust in such measures involves a basic 
faith in human nature and self-analysis. It is the 
confidence that customers are valid judges of the 
quality of merchandise they purchase, that pro- 
fessors can judge the effectiveness of a lecture, 
that students can accurately rate teaching effective- 
ness. etc. It is the confidence that when quantita- 
tive objective data aie not available we are better 
served by a subjective qualitative measure (couch- 
ed in quantitative terms) than by no measure at 
all. 
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Obviously, the collection of these testimonies 
will require administering opinionnaires to the tar- 
get groups periodically. Facsimile questions for 



such opinionnaires are cited in the table as exam- 
ples only. Wording must be detailed by others with 
more expeitisc. 



Table 1. Measures for Educational Goals 



Goal or Objective 



Suggested Measure(s) 



A. Learn to Feel 



B. Learn to Retain Facts 



C. Learn to Think 



D. Learn to Decide 



E. Learn to Act 



F. Learn to Learn 



O 




1. Student Testimony* (e.g., student's answer 
to question, "Has college Increased your 
sensitivity to the problems of others and 
your appreciation for classical music by: 
‘a great deal,' 'a modest amount,' 'not at 
all,’ or ‘decreased’?") 

2. Student Testimony* (e.g., "While in college 
has your factual knowledge: 'more than 
doubled,’ ‘increased some,’ 'remained 
about the same,’ or 'decreased'?") 

3. Difference between percentile rank on 
GRE (or substitute exam) and percentile 
rank on SAT or ACT. 

4. Difference between “before" and “after" 
reading-listening test on totally new ma- 
terial. 

5. Student Testimony* (e.g., “While in college 
your logic and reasoning abilities have: 
’more than doubled,’ ‘increased some,’ ‘re- 
mained about the same,’ or ‘decreased’?”) 

6. Difference between “before" and “after" 
score on Miller Analogy Test. 

7. Student Testimony* (e.g., “While in college 
your ability to choose alternatives, to reject 
some life styles, and to choose wisely: 
‘more than doubled,’ ‘increased some,’ ‘re- 
mained about the same,’ or ’decreased’?”) 

8. Difference between "before” and “after” 
item analysis of College Student Question- 
naire. 

9. Student Testimony* (e.g., "While in college 
has your effectiveness in communicating 
ideas to others: ‘more than doubled,’ ‘in- 
creased some,’ ‘remained about the same,’ 
or ‘decreased'?") 

10. Student Testimony* (e.g., “While in college 
have you increased your ability and you' 
motivation to learn on your own, to be a 
self-starter by: *a great deal,’ ‘a modest 
amount,’ 'not at all,’ or ‘decreased’?”) 
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11. Difference between “before" and “after" 
item analysis of College Student Question- 
naire on questions such as, "How many 
books did you voluntarily read in the last 
six months?” 

12. Difference between “before" and "after" 
score on an artificial language test. 

G. Choose a Career 13. Student Testimony* (e.g., “In selecting a 

career, attending college helped: ‘a great 
deal,’ 'a modest amount,’ ‘not *»♦ all,’ or 
'negative influence'?”) 

14. Per cent of students who Identified or al- 
tered specific career goal while in college, 
as measured by “before" and “after” sam- 
plings. 



H. Gain Admission to 15. Student Testimony* (e.g., “Did college 

Next Stage in Career move you along toward your career goal: 

‘a great deal,’ ‘a modest amount,’ ‘not at 
all’ or ‘negative influence’?") 

16. Change in per cent declaring graduate 
schoul (or medical or law schools) inten- 
tion “before” and "after." 



I. Develop Skills of One’s 
Chosen Profession 



J. Earn a Living 



K. Fulfill Society’s 
Manpower Needs 



17. Student Testimony* (e.g., “While in college 
your proficiency in the skills of your chosen 
profession: ‘more than doubled,’ ‘increased 
some,’ ‘remained about the same,’ or ‘de- 
creased’?") 

18. Difference between "before" and "after” 
percentage of students passing specific 
professional examinations such as the Bar 
Exam or the CPA Exam. 

19. Number of pages published by graduate 
sctiool graduates in scholarly journals. 

20. Difference between Cartter rating of school 
of undergraduate enrollment and of first 
job (applicable to graduate students only). 

21. Student Testimony* (e.g., “While in college 
my ability to earn an acceptable living: 
‘more than doubled/ ‘increased some,’ ‘re- 
mained about the same,’ or ‘decreased’?”) 

22. Ratio of starMng salary of age cohort of col- 
lege gradua* ,s versus matched pair (by 
ability and background) cf persons who did 
not attend college. 

23. Student Testimony* (e.g., "While in col- 
lege my ability meaningfully to contribute 
to the manpower needs of the nation and 
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society increased by: ‘a great deal,' ‘a mod- 
est amount,’ ‘not at all,' or 'decreased'?") 

L. Identify New Phenomena 24. Faculty Testimony* (e.g., "During last year 

your most significant discovery will desir- 
ably alter the course of history: ‘a great 
deal,’ ‘a modest amount,' ‘not at all,’ or ‘in 
a negative way’?") 

2t\ Number of do, Jars received f om new pat- 
ents, 



M. Synthesize and Sum- 
marize In New Ways 



N. Communicate New 
Knowledge 



28. Faculty Testimony* fe.g., “During last year 
did your research and study result in a new 
synthesis or summary that will desirably 
alter the course of history: ‘a great deal,’ ‘a 
modest amount,’ 'not at all,' or 'in a nega- 
tive way’?") 

27. Faculty Testimony* (e.g., "Your thinking 
and writing during the last year has stimu- 
lated professional colleagues: 'a great 
deal, 'a modest amount,' 'not at all,' or 
‘negatively’?") 

28. Number of pages published in scholarly 
journals and books during the last year. 



0. Relate Knowledge to 
Societal Problems 



P. Carry Out Societal 
Project 



29. Faculty Testimony* (e.g., "During the last 
year personal consultation assisted in the 
solution of important societal problems: ‘a 
great deal,’ ‘a modest amount,’ 'not at all,' 
or ‘a negative effect’?") 

30. Number of days spent consulting (paid and 
unpaid) last year. 

31. Faculty Testimony* (e.g., "During the last 
year my hours devoted to implementing a 
society-aiding project contributed to the 
success of that project: ‘a great deal,’ ‘a 
modest amount,’ ‘not at all,’ or 'nega- 
tively'?") 

32. Number "graduated" from funded training 
programs. 



Q. Evaluate Effectiveness 
of Current Attacks on 
Problems 



33. Faculty Testimony* (e.g., "During the last 
year my role in evaluating the effectiveness 
of various projects designed to better so- 
ciety was: ‘a great deal,’ ‘a modest amount,’ 
‘not at all,’ or ‘negative’?") 



R. Increase Dignity and 
Self-esteem of Faculty 



34. Faculty Testimony* (e.g., "During the past 
year my image of myself and my colleagues 
has grown: ’a great deal,' ‘a modest 
amount,’ ‘not at all,’ or ‘decreased’?") 
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35. Percentage change in compensation di- 
vided by percentage change in income for 
entire U.S. 



S. Increase Student Enjoy- 36. Student Testimony* (e.g., “While in college 
ment While in College you had: 'a great time, 1 ‘a pood time/ ‘little 

or no enjoyment,’ or 'a miserable time’? 1 ’) 



37. Per cent intending to donate after college 
“before enrolling” versus per cent intend- 
ing to donate (or donating) “after gradu- 
ating.” 



T. Provide Benefits to 
Town Citizens 



38. Faculty Testimony* (e.g., “During the past 
year town residents not directly associated 
with the university benefited from toe uni- 
versity’s presence: ‘a great deal/ ‘a mod- 
est amount,’ ’not at all,’ or ‘negatively’?”) 



39. Change in number of part-time students 
enrolled. 



40. Change in number of public lectures and 
performances given. 



•Student (faculty) testimony might include not only the assessment of self-growth but also 
the growth of the student’s (lacutty’s) best friend. Thus, an alternate to the question at 
measure one might be "Has college Increased your best friend’s sensitivity . . . ? M 



Forty measures are listed as examples. More 
can and should be added to the laundry list. Then, 
each group using the model should pick and choose 
the measures appropriate to its need and bias. A 
group of faculty may wish, for example, to weight 
Measure 35 (change in faculty salaries) very heav- 
ily and to assign zero weight to Measure 39, either 
because it is unavailable or thought to be unre- 
liable. In a different way a group of legislators 
may wish to weight Measure 25 very heavily and 
to assign zero weight to Measure 19. The beauty 
of the model is its flexibility. It expands and con- 
tracts according to the availability of data and its 
reliability. It changes emphasis according to the 
peispective and purpose of the user. It allows the 
ordinal measurement of changes for any time 
period and for the most micro as well as the 
macro of units. 

To highlight some of the differences in perspec- 
tive on output, in Table 2 I venture the dangerous 
game of speculation, speculation about the weights 
that would probably be assigned to the various ob- 



jectives, and then to the measures by the four in- 
terest groups in our conference matrix. The weights 
assigned to the objectives are “the ideal set.” The 
weights assigned to the measures are “the ideal set 
tempered by differing confidence in the measures 
available.” For example, by faculty the Objective, 
“learn to feel,” is weighted ten but the Measure 1 
is weighted only five. This means that faculty have 
less confidence in the ability to measure “learn to 
feel”-type output than in measures of other types 
of output. 

Generally, the faculty is characterized as plac- 
ing greater value on long-range objectives, on pro- 
fessional-disciplinary objectives, on personal re- 
muneration and satisfaction, and on its own testi- 
mony. Students emphasize professional-skill goals 
and evidence great confidence in their own testi- 
mony. Legislators emphasize hard measures and 
outputs that increase the gross national product. 
Private citizens mix the objectives and biases of 
the three other groups. 




Table 2. Estimated Weights Assigned Educational Goals and Their Measures 



35 



^ u 

H £ 
n W 
32 
£ 



g LO 
£ 



o 

3 lo 



o 

O) 



o 

CO 



| O jn 

I ^ ^ 

<o 



g LO o 
£ 



CO 



o 

LO 



o 

!0 



o 

CO 



s 



LO 

CM 



o 

CO 



8 



o 

O) 






o o 

CO CM 



O LO 

co r- 



o o 

T- CO 



o O O 
g CM co 



a 






o o 

T" CM 



o o 

LO CM 



O O 
t- CM 



o o 

CM LO 



o o 
T- CO 



o o 

LO 






o o 

LO CO 



o o 
LO CO 



§ 



8 



o o o o 
CO CO 



o o o o 

LO LO 






o o 
co in 



o o 

CO CM 



o 

CO 



o 

o 



8 



o 

CM 



o 

8 



[ fl 

w 

> 

p 

u 

w 

»-> 

M 

o 

c u 
o 

t/i 

w 

pc 

3 

lO 

< 

w 

2 



C /5 

w 

> 

p 

u 

w 

►*> 

CQ 

o 



c 

o 

e 

to 

CD 



c 

CD 

TJ 

3 

4-* 

CO 



c' UJ 
o cr to 

EOj? 

to co c 
a> > 0) 

H to 

£ < □ 
§<?v 

"2 ^ « 
2 o 8 
w < cc 



a 

c 

c 

o 

to 

CD 



>* 
c 
o 

E > 

* o to 
— CD 2 



>> 2 >> 
c O c 

o ^ o 

E § E 

4 -» 1-1 4 ^ 



CO 

CD 

I- 



0 — ; u; _ 

H j!j f- (/) 

2 < 2 o c 

•§ * s ® « 

3 = 3 75 3 

35 5 55 o 



a) 



2 

0 

c 

c 

.9 

*■£3 

CO 

CD 

c o 

O 4- 

c c 
.E CD 
*o 

3 



CO 

(D 

h- 

CD 

CD 

<0 

3 

a) 

c 

CO 



eg 



5 o 

CO O 



CCJ 

o 

o 

u 

CD 

*■! 



r* CM CO to CD N- CO O O t- 



o 

£ 



0 


0 

tf 


c 


o 


4 -» 


0 


0 


!E 


0 


o 


LL 


a: 


H 


o 


< 


O 


O 


o 


o 


o 




4 -» 






■ 4 -* 


c 


c 


c 


c 


c 


cc 


c 5 


c 5 


CO 


<5 


0 


0 


0 


0 


0 


-i 


-J 


-j 


-J 


-J 



00 __ V 

CD Ctf r- 
« 

■2 £ s 

“5 E 

. <r 

£ 2 

3 

2 

o 

H 

09 
3 
CO 



c 

o 

E 

CO 

CD 

H 



O 



o 0 

2 0 
co 0- 



o 

o 

SI 

o 

CO 

S' I 

1 5 
E T3 

« 5 

a> a 



<*-» C 
C CD 
CD O 

■o 

.3 CD 
CO 0- 



4 -* K 



>> co 

c E 
o S 
E x. 
Ui 

co 

a) 

■ CD 
N 



rt 



to 

o> 

c 

’<3 

0 

a: 



■a ° 

2 a 
CO CO 



CO CD 
0 t: 
cn t 
0 0 
cl a 



>> 

c 

o 

E S' 

to *2 
0) <3 
f— CO 

C g 

CD .5 

■o tr 

3 CO 



>» 

c 

o 

p 



0 

H 

4«* 

c 

0 

■O 

3 



CO CO CO 



CO ^ LO CD 



CO 

3 

O 

c 



0 

0 

L_ 

CO 

O 

co 

0 

w 

o 

o 

SI 

o 



K CO O) O 
^ r- r- CM 



CM 
CM CM 



0 

O) 

2 

co 

« 

0 

z 



0 

> 

o 

2 



5 

CO 

a 

o 

0 

> 

0 

Q 



o> 

c 

> 

Jj 

CO 

c 

L. 

CO 

Ui 



v> 

s 

Ui 

fc 

a 

CO 2 
CM CO 



< 

H 

2 

OQ 

3 

CO 



CO 

■o 

$ 



0 

s 

o 

a 

c 

CO 

2 



3 

LL 



Q 

til 

z 

o 



TE UO 

C /5 til 

< s 

j 2 o 

t W 

X »-s 
O tt 

S O 



goo 

.£ r- LO 
Cl 



So 

g r- CM 



5 P 



CO 



8 



o 

r- 



o 

LO 



o 

co 



o 

LO 



O 

CO 



o 

o 



o o 
a> oo 



o o 
co oo 



o o 
co CO 



o o 

9 § 



R 8 

eg 



§ £ 

3 00 



o 

CO 



o 

CM 



O 

CO 



o 

CM 



O 

CM 



O 

CO 



8 



O 

CO 



o 

o> 



LO u> 

a> tr 



o 

to 



O o 
<0 £ 



3 O 

ir 



o 

CO 



o 

o 



8 



o o 

CO o 



o 

CM 



o 

CO 



o 

o 

CM 



O 



a 3 . 




'Applicable only to graduate students. 



36 



o o m o o o 



CM 



o w oo mm o o in o o m mmo q 

CM N CM ^ r C\J O t- CM r- d 




S o o in oo *> oo o in in o mo om mom 

grCO r rr N ( 0(0 rP) CO J; CM r- CM 



m 



o o 
N 8 



■o m m m mo £ mo mm m © oo mm mmo © 0 



© 5 

o 



3 m m o o m in mo mo m g oo oo omm g g 

y CM CM CM CM 00 « t- N O O CM r (Q 2 



Uu 

O 

H 

2 

id 

5 

id 

DC 

D 

3 

id 

£ 



Vi 

nJ 

o 

O 



>> 

c 

o 

E </> 

*5 c 
0 ) 0 
<D •£ 

f I 

O ^ 

LL *0 

m 

CM CM 



l 

z 

V— 1 

<D 

> 

o 

o 



>> 

c 

o 

E 

oo 

0) 

H 



$ 

<D 

z 



>s 

II 

(/] ~ 
0 ) X> 

if 



§ 

£(D S 
3 3 0 £ 

0 O 05 £ 

0 C 3 CO "" 

LL LL CL 



CD N- 00 
CM CM CM 



$ 

0 

z 

0 



8 | 
oo 
0 



3 

E 

E 

o 



O o I 

.§2 In 
«1 55 S! 
£0. 



OC 

£ 



I 

n 

3 



>X 

c 

o 

E 

CO 

0 



co o 



£ 

0 

z 

0 

0 

0 

cr. 



2 
CL 
+■» 

3 

o 

£* 2 

k. -* 

0 

o 



£0 £>.C £ gj 

3 00 3 C 3 </> 

O >> O 05 

0 «S 0 il 0 fi 

LL O U. H LL h 

OO O t- CM CO CC 
CM CO CO CO CO O 



</> 

•J 

2 

o 

CO 

3 

^ CO 

o 
0 



o 

>>5 

c £ 

?o 

c a 

Q) O 

J-O 

££ 
3 3 
O O 
03 0 
LL LL 



>* 

C 

£ o> 
E c 

p 

» 4-< 

4- c 

C Q 

0 O 
5 0 

CO CL 



c <0 
^00 
>N C k- 

c E 3 

n ~ 4-« 

c O 0 
fc t: 0 
\3 C _J 

S UJ '5 

H 0 

*.§fc 

~ 

3 c e 

0 0 3 

LLQL z 



ts 




</> 


0 


0 


& 


c 


0 


'2 

Cl 


3 

O 

0 


0 

-0 

3 


E 

0 


0 


LL 


CO 


H 


0 


£ 


44 


*3 


3 


c 


O) 


0 


0 

> 


Ol 


0 


C 

0 


LU 


5 


O 


CO 



m- m co h* 00 00 o 

CO CO CO CO CO CO M- 



E | 

ft H 

O o 



</> 



< 

__ OC 

< o 

o 

0 

3 

</> 



Q 

Id 

2 

0 

8 

< 

v> 

f- 

X 



18 

> 



a 

iS in 

> CM 
£ 



m 

CM 



g o 

-8 CM 
3 
« 

□ m 

o r- 

£ 



mm w o o o g o m m 

CM CM ^ m oj ro 0 <*— ru r- 



m m 

CM CM 



o o 

T- CM 



m cm co g 



r ie 



o m 

W T- 



15 



o 

CM 



m 



8 8 



CM 



m 



R 



m 

CM 



mo ©m m m v> o o o 

CM m irt r- « 10 CO CM 



o o 
in o 
o 



o o 
u> g 



§ § 

1 - © 



g § 
« © 




The figures here are only guesstimates. The 
model is not ideal but it is a start. A department 
chairman may begin evaluating output immedi- 
ately by applying a simple four-variable model to 
his graduating seniors. He could arbitrarily weight 
the four variables or, better, seek the consensus of 
his depattinent. Possible weights might be: 



Measure 3, ACT-SAT vs GRE 300 

Measure 5, student testimony 400 

Measure 16, % grad school 200 

Measure 28, pages published 100 



Total 1000 



Each year, more data and more experience 
with each measure will stimulate adjustments in 
the model. By try and cut, a workable model 
evolves. 

Similarly, a legislative review committee or a 
citizens 1 group may structure a model unique to its 
needs and biases. If, at first, the data desired are 
unavailable, “next best" measures can be inserted 
until the preferred data are available. The group 
might, for example, initiate a model with only two 
measures (say Measure 18, specialized exams; 
and Measure 22, starting salaries). In the second 
generation model, Measure 18 might be discarded 
and Measure 17 plus Measure 2 added. 

Accrediting teams, visiting examiners from gov- 
erning boards, legislative review commissioners, 
and other evaluators may ask an institution (e, g., 
a university) or a group of institutions (e. g., a 
university system, a mental health system, a high- 
way system) to choose its own measures, use its 
own weights, and submit an evaluation of progress. 

The value of this approach is its flexibility and 
the fact that it may be implemented immediately. 

f am proposing a kt's-get-started-now scheme 
for evaluation. The scheme is far from ideal. It will 



be applied differently by different groups; it meas- 
ures only change in output and not absolute worth; 
it relies on untested measures — but it is a start! 
There are compelling reasons for higher education 
to start evaluating output, e\en if it must be by try 
and cut. 

The proposed evaluation scheme is based on 
Diagram 1. 

Educational outputs (see Diagram 2) are both 
the appreciation of human capital (whole man, 
pool of knowledge, specialized man) and immedi- 
ate enjoyment (public service, joy of producing). 
The total contribution of higher education is the 
sum of these outputs, as shown by the Equations. 

Stimulating the growth of incoming students 
are three basic environmental forces (Diagram 3): 

(1) Human resources: for example, size and 
quality of faculty, size and quality of sec- 
retarial help, size and quality of adminis- 
trators, size and quality of student body. 

(2) Physical resources: for example, build- 
ings, educational media system, computer, 
library, location relative to other colleges 
and a central city. 

(3) Tactics and Methods (of relating resources 
to input): for example, organization of 
curriculum, governance structure, teach- 
ing methods 

To evaluate higher education's successes, 
measured output must be contrasted to the cost 
of providing the environment. Environment is 
created by committing resources; the measurement 
of this commitment is relatively simple. Historical- 
ly, the problem has been measuring the evaluating 
output. This paper proposes a scheme to solve 
that problem. 



Diagram 1. The Simple Model 



Input: Value of 
“Goods in Process” 
When Received by 
Higher Education 



Environment: 
The Production 
Process 



Output: Value of 
“Goods in Process" 
When Shipped Out 
by Higher Education 



Input passes through the environment to become output. The 
objective of higher education is to add value to the input (stu- 
dents), as it passes through the environment (college). 



38 



Gross Value of Input 



Diagram 2. The Basic Model 

Gross Value of Output 




Environment 






Public 
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Equations : 



Net Value 
of Output 



Gross Value _ Gross Value 

of Output of Input 



Net Value _ Growth in , Growth in , Growth in , Public Service , Joy of 

of Output Whole Man Specialized Man Pool of Knowledge + Produced + Production 



Diagram 3. Components of Environment 
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"/ have a feeling.. . that many high school students look on 
college as continuing the most unsatisfactory aspects of 
their high school experience, while at the same time provid- 
ing no qualitative change in their lives." 

"By what means can we educate ourselves about the best 
method of education without having presupposed the an- 
swer to the very question we are asking?" 

"We should examine the cost of the Implied change of the 
educational process from an emphasis on the transmittal of 
knowledge to an emphasis on the maturation of the Inquiring 
mind... It Is obviously premature to say whether the grow- 
ing need for opening up inquiry really constitutes a major 
revolution In the Institutional system." 

"If it Is true that most Ph.D.’s do not do research after receiv- 
ing their degrees, then surely the output of Ph.D. programs 
is poor. I believe that most of our graduate programs... 
simply tend to produce people whose continued Interest in 
Inquiry has been killed off In the process of the program." 




R* on E: Some Suggestions for Research on the Role 
of Research in Higher Education 

C. WEST CHURCHMAN 



The purpose of this paper is to discuss the role 
of research in the educational process, especially 
at the higher educational level, but not necessarily 
restricted to this area of education. My main effort 
will be to try to suggest some areas where edu- 
cational research on the research process could be 
quite helpful to the future of higher education. 

I need to begin this discussion with a number 
of apologies and explanations. 



Determination of Goals Emphasized 

First, I realize that I’ve probably been asked to 
deliver this paper because I’ve written on the “sys- 
tems approach.” The paper, however, emphasizes 
one aspect of this approach, namely, the deter- 
mination of the goals of the educational system 
and not the efficient ordering of components and 
resources to attain given goals. In other words, I 
have a strong bias in the subject of educational 
planning toward a continuing re-examination of 
our educational goals. I realize that the attempt 
to define the goals of education was at one time 
very popular and has since entered into a period 
of disillusionment as we discovered how difficult 
it is to define these educational goals in a man- 
ner which would make any difference in the edu- 
cational process. I think the disillusionment itself 
was mistaken, just as a disillusionment in the at- 
tempt to define national goals would be a mistake. 
Obviously, an area so rich and difficult as the 
definition of human goals demands, like any life 
form, a growth through immaturity to some kind 
of mature status; and the maturation period may 
be very long, indeed, as I am convinced it will be 
in the case of educational goals. Simply to be dis- 
appointed because in a relatively short period of 
lime we have not been able to reach a reasonable 
consensus on educational goals is certainly no rea- 
son to give up the enterprise. 

Second, I need to say some things about the 
“outputs” of higher education in the context of 
goal-setting. As will be seen in the second part 
of (his paper, Fm convinced that (here is a task 
of reflection that needs to be undertaken before 
we can feel satisfied with any specific proposals 
for outputs. Part of this task consists of trying to 



decide whether we should regard the primary out- 
put to be the transfer of knowledge, or rather the 
maturation of each individual’s style of inquiry. 
I think we get radically different analyses of sys- 
tems of higher learning depending on how we re- 
spond to this issue. 

Third, it should be pointed out that the paper 
really concerns itself with research at three differ- 
ent levels: 

(1) There is the research carried on by the 
student, and the relationship of this research to 
his other educational and social activities. 

(2) There is the research undertaken by the 
educational researchers who. are trying to examine 
some issue in (he connection between the role of 
the student’s research and his educational goals. 

(3) There is the research of this paper itself, 
which, as I have pointed out, is somewhat limited 
with respect to background literature materials. It 
is also strongly biased by my own research prefer- 
ences at the present time and, namely, the use of 
research to study large social systems. This bias 
means that a good deal of what I will have to say 
will not be restricted to the research process itself, 
either at the student level or at the educational re- 
search level, but to the relationship of this process 
to other kinds of activities occurring in society. 



Concept of Research Explained 

I must also express a bias regarding the con- 
cept of research. This can best be explained by a 
very broad definition of research, namely, that re- 
search is a set of human activities where the activi- 
ties are intended to produce knowledge and, in 
particular, a knowledge of a more effective means 
of accomplishing socially desirable ends. You will 
note that in this definition I have made no reference 
to such things as control of variables, statistical in- 
ference, and the like, because I regard these to be 
possible tactics and strategies of the individual 
whose intent is to acquire knowledge. These strate- 
gies may, in fact, not be desirable in many contexts. 
They are, of course, highly desirable when it’s pos- 
sible to conduct the research within experimentally 
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controlled situations, and when there is an exclu- 
sive desire to learn exactly wliat elements of nature 
cause other elements to behave in a certain manner. 

My bias is that I believe that educational re- 
search and, in particular, educational evaluation 
makes a mistake if it attempts to wed itself rigidly 
to the strategy of precision and control. There's 
an immense amount of knowledge acquired by a 
human being without the use of carefully con- 
trolled experiments. Take for example, the process 
of learning how to drive an automobile, which, for 
each individual and within that context, falls under 
my definition of research. It could scarcely be said 
that the learner carefully controls the variables as- 
sociated with his hands, feet, and eyes, and learns 
which particular motions cause various events to 
I occur in maneuvering an automobile. There’s much 
’ to be said on the benefit side for carefully con- 
trolled experiments, but there is also the cost of 
rigor to consider. In the terms of the systems scien- 
tist, we require a ‘‘cost-benefit” analysis of our re- 
search strategies. In order to keep educational 
evaluative research from becoming unnecessarily 
trivial, it may be necessary to give up some of the 
traditional concepts of control. At the same time, 
of course. I'm not saying that loosely designed re- 
search is good in and of itself. What obviously is 
required in the field of educational research is some 
research aimed at trying to evaluate strategies and 
tactics for various kinds of problems. For exam- 
ple, we need to know a great deal more about the 
circumstances under which carefully controlled ex- 
periments have a positive net value in the educa- 
tional process. We also need to learn a great deal 
more about the conditions under which simulation, 
e.g., on computers, is a useful way of learning 
about the educational process. 

Consequently, the first area of research I would 
recommend for study is a “systems approach” td 
educational research, i.c., an examination of the 
tactics and strategies of such research systems and 
a systematic way of trying to evaluate where these 
strategies and tactics seem best and where they be- 
come less than adequate. 

Research by the Student 

With these apologies and suggestions behind 
us, suppose now we turn to the first level of re- 
search, namely, the research carried on by the 
student in higher education. We can begin by ruling 
out considerations of certain aspects of research 
In the student's life. I don’t doubt that the role of 
research in graduate education, especially at the 



Ph.D. level, is well established within many disci- 
plines. It is not apt to change radically in the com- 
ing years I think it is a plausible hypothesis that 
in many disciplines, especially in the physical sci- 
ences, the role of research in the student’s program 
is well embedded within the policies of the par- 
ticular discipline. To be sure, there may be a 
number of questions that one would want to ask 
about specific policies of institutions with respect 
to the rigidity of course requirements, the preclu- 
sion of the student from research programs until 
much too late in his curriculum, and so on. In this 
paper, however, I’m not addressing myself to the 
problem of the role of research within well estab- 
lished graduate curricula. 

There are two areas which I would like to 
examine. One is the rather specific role that re- 
search might play in the teaching process within 
higher education. The second is concerned with 
the overall goals of the educational process and 
specifically with the contrast between “knowledge 
transfer” and “maturation of an inquiring style.” 

Turning to the first of these, it is perhaps use- 
ful to review the debate that has been waged about 
the relationship between teaching and research in 
institutions of higher learning. 

The traditional approach can be rather easily 
stated. It argues that one of the principal tasks of 
most institutions of higher learning, especially at 
the undergraduate level, is to provide education to 
a large number of students as economically as pos- 
sible. The argument stipulates that the only means 
of accomplishing this goal is the traditional method 
of teaching in classrooms either to a large number 
of students who are subsequently broken up into 
quiz sections or to a large number of sections, 
many of which are run by young teaching assist- 
ants. Research, the argument goes on, demands in- 
dividual attention because every research project 
is highly specific; if the research is to be performed 
in an adequate manner, the younger student re- 
quires very careful supervision on the part of some- 
one knowledgeable in the field, e.g., an associate or 
full professor. It is not feasible, therefore, to ex- 
pect that a large number of students can be taught 
by means of an expanded research program in 
which undergraduates are involved actively in do- 
ing a significant amount of research. Furthermore, 
research is highly specific, because it must deal 
with a very concrete aspect of the natural world. 
But the purpose of much of undergraduate edu- 
cation is to provide the student with as broad and 
general an educational base as possible. Of course, 
research should be an integral part of the cur- 
ricula when the student is being prepared for a 
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life as a scientist or an engineer, i.e., in profes- 
sions where research itself plays an integral role. 
But for the general student a significant concen- 
tration on research deprives him of the kind of 
broad education which our undergraduate pro- 
grams are supposed to provide. Finally, the argu- 
ment concludes by pointing out the other de- 
ficiency of research, namely, that it is in conflict 
with the teaching program in terms of faculty time. 
The argument stipulates that faculty are rewarded 
for their research output, especially in large uni- 
versities, and that this reward system in effect re- 
quires that the faculty concentrate on their re- 
search efforts to the neglect of hours that might be 
spent in direct contact with undergraduates. 

Research As Teaching Method 

The other side of the argument can best be 
expressed in its strongest form in the terms of 
one of my earlier teachers, Professor Henry Brad- 
ford Smith, who was a strong proponent of the use 
of research as a teaching method. According to 
Smith, the ideal college would be one in which the 
undergraduate from the beginning works with a 
professor in developing a research project which 
will occupy his attention during his four years. As 
a result of the design of a specific research project, 
according to Smith’s concept, the undergraduate is 
led naturally into all the ramifications connected 
with his research. He will sense the requirement 
to write well and even dramatically, to understand 
the histOiical background of his research effort, to 
delve, if necessary, rather deeply into the mathe- 
matical and statistical methodologies required to 
bring his research to fruition. In other words, he 
will attain an understanding of the relationship 
of his own research program to the larger system. 
In such a way, thought Smith, the student would 
be led to the kinds of “general education” which 
has been so strongly advocated in past decades. 
But instead of being fed general < ducation in a 
piecemeal fashion, his education would become 
meaningful and relevant to him. As a historical 
point, I might say that Smith’s concepts of edu- 
cational relevance were stated some forty years 
ago. The notion that a great deal of so-called gen- 
eral education is irrelevant is by no means news to 
many of the teaching community. 

We can see that the issues involved in the 
debate are by no means so specific that one could 
conduct a test that would once and for all settle 
the matter. This situation is quite characteristic of 
the major issues that concern any large scale social 
system. Immanuel Kant in his Fundamental Prin- 



ciples of the Metaphysics of Morals states the mat- 
ter superbly. In his discussion of what he calls the 
prudential imperative, which today we would 
translate into the concept of policy, Kant argues 
that the basic issues cannot be settled in any final 
way because our knowledge depends on specific 
empirical tests; whereas the settlement of basic 
issues involves the total system of a person’s life 
or many persons’ lives. 

Kant’s point is well taken in connection with 
the issue we are now discussing. The question is: 
What policy should be followed in undergraduate 
education vis-a-vis research and teaching? Under- 
graduate education is only a part of the total life 
of the individual; although it is probably accurate 
to say that in many cases the four years of under- 
graduate life have tended to be rather decoupled 
from the rest of the individual’s life, e.g., his high 
school education and subsequent career after 
graduation fiom college. This does not mean that 
research has no role in clarifying the issues. It 
simply implies that research by itself cannot ever 
settle the debate. The role that research will play 
in the present discussion is to identify one or more 
of its weakest elements of the world view of each 
side. 

In this case, as I see it, one of the chief weak- 
nesses of the traditional world view of the under- 
graduate educational process is its claim that the 
traditional method of teaching a student in a class- 
room is the most effective way of handling student 
education. This position seems especially weak in 
the case of what might be called the methodological 
sciences, i.e., statistics, mathematics, logic, etc. The 
methodological sciences should also include English 
composition and what used to be called rhetoric, 
i.e., the ability to express oneself clearly in the 
spoken word, and, perhaps, other disciplines like 
foreign languages and even historical methodology. 

The kind of research I have in mind would 
entail a study of the extent to which a student 
learns his methodology in the context of research, 
compared to his learning in the more traditional 
classroom situation. And in this connection I 
should make it clear that when I use the term, 
“student research,” in this context, I’m thinking 
of a student who’s studying a real-life problem, 
say, a problem of the planning of an organization 
such as a university library or a health center or 
a business firm. To make the point clear, it is 
helpful to distinguish between the “exercise” and 
the “problem.” Exercises that are found at the 
end of texts in statistics, operations research, etc., 
almost always provide the student with an oppor- 
tunity to “apply” what he has learned in a given 
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chapter; but they may not be helping him develop 
his methodological capabilities in real-life situa- 
tions. Exercises at the end of chapters can be 
describ'd as tasks where the basic data and system 
constraints are eivon by an external authority, in 
this case, the textbook or the teacher. In the real- 
life situation, such basic information and con- 
straints have no dear authorities. The student 
learns that there are differences of opinion within 
the organization as to what things are relevant or 
what constraints should be placed on a problem. 
Indeed, many students who have received a 
thorough training in the methodologies, when faced 
with problems in real-life organizations, often ?et 
extremely frustrated because there is a lack of au- 
thority. My point can be well illustrated by a recent 
textbook in management science by Harvey Wag- 
ner called Management Science: Applications, 
What one finds in Professor Wagner's book is a 
series of exercises in which the student is provided 
all of the basic elements of a solution. It may be 
a disservice to the methodology itself to claim that 
such exercises ill prepare the student for “ap- 
plications.” 

Which Input Produces Best Results 

Thus, in the educational research I have in 
mind, an attempt would be made lo determine 
whether a student learns better how to use statis- 
tics. mathematics. English writing, etc., outside the 
classroom preparation within the context of a real- 
ise research problem; or whether a certain amount 
of formal teaching and real application Is better; ot 
whether the entire subject matter can be learned 
within the classroom. Phrased in terms of output, 
the question is whether an input of classroom hours 
or an input of research activity produces more 
ability to solve (he social problem as judged, say. 
by a group of experts, or the student himself. 

It goes without saying (hat the research f have 
in mind would be done with psychological corre- 
lates in mind. Obviously, some people do have a 
capability of storing a vast amount of information 
about techniques, and it may very well be ln>e that 
such persons can learn just as well or even better 
in the classroom exposure than they would within 
the real-life research. It also goes without saying 
that a "control group* should be included in such 
an experiment, where the student is essentially self- 
learning. i e., studies out of the text on his own 
with minimal consultation with a teacher. 

A great weakness tn the position that research 
is the best way to learn is the assumption that this 
method can he used with a large number of stu- 
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dents and a relatively few' number of faculty mem- 
bers. In other words, it looks as though a much 
greater input of teacher-hours is required to imple- 
ment the learning-by-rcscarch plan, perhaps a 
totally unmanageable amount of input. The research 
position naturally has to argue that the present 
method of organizing a faculty member's time is 
incorrect. It's clear that the professor's main ef- 
fort should be to attain a face-to-face contact with 
the student in directing him into bodies of informa- 
tion. methods, etc., which will help him to accom- 
plish his research mission. This means that the 
teacher's student-classroom exposure hours would 
be reduced to a minimum. The position can hard- 
ly hold that no classroom teaching is lo be per- 
mitted. Evidently a number of students could 
gather together to study a certain phase of a prob- 
lem where, say, statistical methods arc essential. 
In such an event, the traditional classroom tech- 
nique seems obviously the correct one to follow*. 
What the position must try to explore is the ex- 
tent to which students can be their own educators, 
i.c., the extent to which one student can be suc- 
cessfully used to teach another or whcic a student 
can engage in his own self-teaching. The concept 
is well expressed in Russell Ackoffs paper on the 
"new university" in which he discusses the educa- 
tional organizations broken up into "self-teaching 
centers." 

There U another aspect of this problem which 
needs to be investigated. If we were to attempt to 
change institutions of higher learning into self- 
teaching institutions, then obviously there would 
be a number of political pressures imposed by 
both administrators and faculty to prevent this step, 
which in a way looks like a relinquishment of the 
faculty’s prime responsibility. This is the problem 
of implementation which plagues all planners and 
systems scientists. It is the implementation problem 
in its very strongest form, since the policies being 
considered arc those which appear to strike at the 
very heart of the traditional way of running the 
academic institution. What seems really called for 
in this connection arc seme educational inventions 
which on the one hand preserve the faculty's tra- 
ditional role of freedom and educational leadership, 
but at the same time provide a kind of flexibility 
in which the ^udent can enter into a teaching rela- 
tionship with other students. 

The Search for Breadth a mS Depth 

I'd like to turn now to the second topic of my 
presentation, which concerns itself with the type of 
research which the student might carry on which 
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would provide (he kind of educational breadth and 
depth that is so desperately needed. I’d like to 
return to some remarks that were made earlier 
about the relationship of the college to the other 
aspects of an individual's life, i.e., the linkages 
between the college part of his life and his high 
school and subsequent career. ) have a feeling, un- 
supported by any large amount of information at 
my disposal, that many high school students look 
on college as continuing the most unsatisfactory 
aspects of their high school experience, while at 
the same lime providing no qualitative change in 
their lives. Part of this feeling undoubtedly arises 
because of the ill-stated goals of colleges with 
respect to their educational programs. 

The one common conception of the goal of a 
college can be described in terms of a model of 
the educational process in which the educational 
institution acts as a transferer of knowledge from 
books or from individuals to the student. This 
knowledge, of course, is not just restricted to facts, 
but involves conceptual knowledge of the ways in 
which people should think about issues in a ration- 
al manner, appreciate their environment, the arts, 
and so on. The essential idea, however, is based 
on the transfer concept, so that the faculty as well 
as the library play the role of the storehouse of 
human knowledge; and the student plays the role 
of the recipient of this knowledge. The aim, then, 
is to produce an individual who is knowledgeable 
in the sense that he can respond, say in a conversa- 
tion. to issues familiar to his peers. 

In the remainder of this paper. I would like 
to discuss an altcWivc objective and its corre- 
sponding model for the institution of higher learn- 
ing. This objective of the college is based on the 
model which says that the purpose of the college, 
as well as other educational institutions in an in- 
dividual’s life, is to create a "mood for inquiry.” 
According to the model, this mood of inquiry must 
fit into the individual's personality and life style, 
i.e., to his psychological being. Consequently, the 
emphasis is more on the activity of inquiry, as it 
becomes meaningful to the individual, than it is 
on any specific informational content that he should 
receive according to certain prescribed educational 
standards. Also, the purpose is not to transfer 
knowledge per re. because now knowledge becomes 
only a means of inquiry. Rather, the purpose is to 
create within the individual the motivation toward 
inquiry, as one of his life functions. 

To summarize, the fundamental characteristics 
of inquiry are the following: 

a. the motivation to learn 
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b. openness to change of fundamental assump- 
tions 

c. the development of inquiry as an integral 
part of the individual's total life style 

As I suppose is obvious, it is no easy matter 
to define these characteristics in terms of some 
measurable "output”; nor is it clear the input- 
output analysis would be appropriate for this mod- 
el. In the transfer-of-knowledge model, on the 
other hand, input-output analysis seems to be very 
satisfactory; not only can one measure, in princi- 
ple, the amount of knowledge transferred, but also 
the quality and possibly the benefit of the knowl- 
edge. In the slyle-of-inquity model, the output 
might be defined, in a longitudinal study, as the 
person's relative interest in continuing to learn; 
but how to make such a concept operational is not 
at all clear. In other words, if one looks at higher 
education through the spectacles of the second 
model, one will have to give inpul-output analysis 
a careful scrutiny to see if it’s the right strategy. 

It would, obviously, be impossible in this paper 
to display all of the different styles of inqu'ry, es- 
pecially if one admits that inquiry is an Integral 
part of (he individual's life end, hence, that the 
style is virtually unique to cac.i individual. It may 
be worthwhile, however, to repeat that aspects of 
inquiry, like consistency, precision, and control, 
do not represent the fundamental characteristics 
of inquiry. These characteristics, to be sure, have 
proved extremely useful In certain areas of science 
and scholarship; but. as I mentioned earlier, they 
have also iended to narrow the scope of the re- 
search and. Indeed, in many instances to make it 
trivial and thoroughly unimportant, as far as either 
social change or the individual is concerned. It goes 
without saying in these days of the proliferation of 
research findings that publication is also not a 
fundamental characteristic of inquiry. There is. 
of course, the need for certain individuals to com- 
municate with others in the process of inquiry; but 
it is doubtful whether the particular technology 
of the printed wx>rd really senes as an important 
characteristic of Inquiry for most people. 

The model that I am discussing, which is re- 
lated to what Kant calls a “whole life model,” 
obviously requires the development of an inquir- 
ing mind at the earliest possible stage, ever, in 
the pre-nursery school, and certainly through gram- 
mar and high school. For example, there seems to 
be no obvious reason why grammar and high 
school students should net. if they feel so inclined, 
express a strong interest in the manner in which 
their school is run, and in the environment of the 
school. 
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Maturation of an Inquiring Mind 

Ii would seem reasonable to assume in this 
model that college, for the person of the years 
from 17 to 21, becomes essentially the locus of 
the maturation process of the individual, especially 
of his inquiring mind. Consequently, the “success" 
of the college program within this model is essen- 
tially to be assessed in terms of whether or not the 
program seems able to bring the student from pre- 
maturation through to the mature individual who 
has confidence in his own methods of inquiry, the 
motivation for inquiry, and all of the humility and 
modesty that go along with the inquiring process. 

It’s also assumed by this model that the matura- 
tion process by no means slops at the end of col- 
lege. After college comes the life process in which 
"higher education" needs to play a fundamental 
role. 

If. in this model, it is appropriate to consider 
"college" as the maturation process, then one “out- 
put" of college is the student’s satisfactory realisa- 
tion that his mode of learning has matured. Such a 
student, in effect, would have learned enough about 
himself to know why he went to college and why 
he should go no longer. The time span to reach 
this maturation might be four years or four days 
or whatever. 

Finally, it's assumed that everyone is Included 
in the total process of becoming a mature, inquir- 
ing mind That includes the faculty, who can no 
longer be regarded as the authorities on curriculum 
and knowledge transfer. The faculty themselves 
are in the process of developing the<r Inquiring 
minds. No amount of depth of inquiry In a particu- 
lar field necessarily completes the inquiring aspect 
of the mind. 

The model assumes that the direction of inquiry 
is a matter of individual style and motivation. \1- 
though. today, there is a great deal of talk of using 
inquiry into matters concerned with the community, 
it is not the inter; of the njodel to claim that such 
inquiry has superiority over other types of inquiry. 
It is dear that the human race at the present time 
knowx very tittle about its communities, both the 
smaller communities surrounding urban and rural 
centers of population, and the larger communities 
of the state, nation, and inter-nation. Our lack of 
knowledge is well displayed i i the fact that we have 
very little reliable data concerning the impact of 
social change. NVe can regud. for example, the 
poverty program in its early years as essentially a 
blind man’s walk; because the planners knew very 
little about the conditions of poverty and the reac- 



tions of the poor class, as well as reactions of other 
individuals who might become interested in poverty 
as a business enterprise. 

The lack of good information is well demon- 
strated in the field of economics, where, although 
we have become reasonably powerful in certain 
theoretical concepts, the basic information is miss- 
ing. 

But the direction of inquiry may be into many 
other areas. Dr. Kilton Stewart has an interesl'ng 
report on the Senoi of Malaya, a tribe which spends 
e great deal of its inquiring lime with dream ma- 
terial from childhood all the way to death. Accord- 
ing to Dr. Stewart, the children are encouraged at 
a very early stage to regard their dream life as an 
important part of the natural world and to try to 
understand its import and to try to control their 
own role within their dreams. The result, according 
to Dr. Stewart, has been a society without crime 
or any other forms of severe social disruption. This 
is the case where the total community has taken on 
an inquiring style in a radical direction. 

in the second part of this paper, the discussion 
has been held at a very general level in order to 
set the stage for more specific suggestions fer edu- 
cational research, which I'll describe within three 
styles of inquiry: the experimental, the systemic, 
and the philosophical. 

The experimental style of Inquiry, as I have 
said, emphasizes control and measurement. It i t- 
tempts to be as precise and consistent as posisble 
and to control the variables in such ways th.t the 
inquirer feels that he is justified in making a sub- 
stantially valid conclusion. It is doubtful whether 
it would be legitimate to call the experimental style 
"objective’’; because, in its broadest sense, objec- 
tivity refers to the ability to gftsp the essence of 
reality; whereas the experimental style in nr. any 
cases docs nothing of the kind. Indeed, the concept 
of objectivity in research that we have inherited 
from the Fifteenth and Sixteenth centuries prob- 
ably needs a far broader meaning when wc turn 
to problems of social change, for example, of the 
sort that would be entailed in the research on the 
maturation process of the inquiring mind. 

Examlna Reward Syatam 

The most obvious experimental study which 
might be made is to examine a revised reward sys- 
tem for education. In the model which claimed that 
the purpose of the college Is to transfer knowledge, 
the reward system essentially becomes the grading 
system. It is taken for granted that the knowledge 
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inherent in a college, e.g., in the faculty’s minds 
and the library, is adequate and docs not need to 
be studied in depth, except at those infrequent times 
when the college itself is evaluated. It is, rather, 
the student's ability to receive knowledge that is 
being transmitted to him, which is taken to be the 
fundamental reward system. The grade that the stu- 
dent receives then represents a siandard measure 
of accomplishment of students within the college 
system. The output can then be expressed as credit 
hours or scores on standard examinations, although 
it is questionable whether either of these is really a 
suitable output measure for the information-trans- 
mittal model, because within this model we are in- 
terested mainly in whether the student can use the 
transmitted information. 

In the “inquiry" model which regards the col- 
lege as essentially promoting the malu ation of the 
inquiring mind, it is clear that since everyone is in- 
volved in the process, the process is to be evaluated 
primarily for the purpose of learning how the pro- 
cess proceeds, rather than trying to grade individu- 
als in terms of their level of attainment, To be spe- 
cific. suppose, for example, a college would enter- 
tain the prospect of an experiment in which there 
would be two student groups, both of them con- 
cerned with the ecological problems associated with 
air, water, and other types of pollution. The first 
group would be exposed to the us^al courses in 
biology, engineering, etc., which are relevant to 
ecological problems. It would go through the nor- 
mal process of consultation with faculty, the stu- 
dent's own individual course papers, and possibly 
a terminal “thrsis." The student would receive 
grades in the usual manner. 

In the second group, each student would work 
out his program in terms of his own inquiring style 
and interest. The faculty would not necessarily play 
a primary role in holding the group together, but 
would let the leadership characteristics develop 
within those individuals who have any Interest in 
this aspect of inquiry. The group would decide on 
the type of personal contact each individual would 
find most useful. There would be no grades asso- 
ciated with the program. Naturally, formal courses 
may be held, because »hcse may provide the op- 
timal method of developing the students' style of 
learning; also, examinations may be held, because 
an examination provides a method of pulling to- 
gether an assortment of things learned. Grades 
might still exist, with »he idea that they preside a 
competitive environment so essential for rrrny 
learners. But no one is required to take the course 
OT the examination, or to compete for a grade. 
Thuj the plan avoids the fallacy that because 



courses, grades, and examinations work well for a 
significant number of students, they should be re- 
quired of all. Students would be expected, at the 
end. to try to evaluate their own learning process 
during the exposure and to point out its essential 
weaknesses. 

Since this is an experimental program, accord- 
ing to its style it needs to come to some kind of a 
conclusion; “What is being tested in this program?” 
is the type of question the experimental mind natu- 
rally poses. It seems almost obvious in this case 
that what is being tested is the ability ,*c the edu- 
cational program both to transfer knowledge and 
to bring about a more mature inquiring mind. The 
evaluators therefore (in this case, the "experi- 
menters") would be engaged in trying to deter- 
mine the extent to w’hich the first group, i.e., the 
group that is oriented toward the transfer of knowl- 
edge. has also brought about a "maturation of the 
inquiring style" in the individual. Consequently, 
the first group, in addition to the standard kinds 
of tests of its ability to acquire knowledge from var- 
ious sources, would also be tested in terms of in- 
terest aroused by the group's program in further 
inquiry, i.c., the kinds of doubts and motivations 
for further learning that have been generated. 

The second model of education that I discussed 
above obviously makes the interest in continued 
inquiry* a fundamental measure of performance of 
the educational process. As a consequence, rather 
than just examining students in terms of what they 
have learned, the second model emphasizes an ex- 
amination of students in terms of what they wish 
to learn or. if the study is a longitudinal one, the 
kinds of inquiry that they go into after a particular 
stage like the one being discussed. But the experi- 
menter’s aim would be to determine the extent to 
which the "transfer of knowledge" goal Is met by 
the second group. In other words. we have two 
strategics of education, and two goals; does the 
experiment indicate that one strategy dominates 
the other with respect to both goal:? 

It is worth noting in this regard that the dif- 
ficulties of defining a suitable output measure may 
be partially overcome if the planners have reduced 
the number of alternatives, e.g.. in this case to lwx>; 
and one alternative dearly dominates the other on 
alt relevant criteria. 

Sy*t*mlc»p a Broader Inquiry 

Now. suppose we turn from the experimental 
approach to the other two. the sjstcmic and philo- 
sophical Systemic inquiry has to give up some 
of the control and precision of experimental in- 
quiry because it reaches out into a broader do- 
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main. It may indeed have to give up consistency 
in its usual sense. If it is to be successful, systemic 
research definitely has to imaginative and spec- 
ulative. Some of the best literature on the future 
of society is systemic, in that the research obvious- 
ly is not trying to establish once and for all what 
the future of society will be like; and in that the 
best reseatchers do use their own intuitions, imagi- 
nations, and speculative abilities to write their 
scenarios. 

In the case of the suggestion that I am making 
in the second part of this paper, the need for sys- 
temic inquiry is clear enough. What the second 
model obviously suggests is an expansion of in- 
quiry as a mode of human living so that literally 
every individual within the society has an intense 
interest in inquiry according to his own style. A 
very fundamental systemic question would be the 
societal implications of such an expansion of in- 
quiry with respect to other social institutions. For 
example, it is perfectly clear that many, if not a ma- 
jority of human beings, arc intensely curious about 
the lives of other people. This curiosity is by no 
means restricted to the feminine side of the human 
race. Ft is very natural for us to be snoopers, (In- 
deed. snooping may bo one of the main reasons 
why young men go into the social sciences.) An 
educational system that expands the natural in- 
clinations to inquiry may. very well, also expand 
the natural inclination to snoop. What would such 
an expansion mean, say with respect to political 
institutions, to family structure, or to “law and 
order** institutions? As often happens, systemic 
inquiry takes us to the door of the house of morali- 
ty. if not Into the interior chambers. Will we have 
to add “consideration of others." or “fairness." 
as constraints on the natural inclination to inquiry? 

Systemic inquiry is absolutely essential as a 
backup of Input-output analysis; for what appears 
to be a reasonable output at one level of society 
may become a horror at another level, e.g.. number 
of automobiles produced, or number of slum 
dwellings cleared. We have to determine where 
the output goes and what it does to other societal 
values when it gets there. By the same token, we 
need to know where the input comes from, and 
who sacrificed what to create it. In this regard, 
“actual dollars spent** may be quite a deceptive 
input, because the dollars spent may have deprived 
some very critical social programs from being 
adequately funded. 

Finally, the systems analyst of rec nt years has 
tried to find some general “scoring system." like 
cost-benefit, to enable him to evaluate alternative 
plans Naturally there has been considerable de- 
bate about the appropriateness of these scores; but 



in my opinion the systems analysis effort is worth- 
while. especially since it tells us how little wc really 
know r about our marvelous suggestions for social 
change. Hence, I’d strongly support a systems 
study of any suggestion for radical change in edu- 
cation. Thus, if my “inquiry” model makes sense, 
wc should examine the cost of the implied change 
of the educational process from an emphasis on 
the transmittal of knowledge to an emphasis on 
the maturation of the inquiring mind. Of course, 
this aspect of the inquiry will get into the details 
concerning the real difference between the two 
goals. Are wc talking about a ma^or change in edu- 
cational institutions or merely a change in emphasis 
here and there? It is obviously premature to say 
whether the growing need for opening up inquiry 
really constitutes a major revolution in the institu- 
tional system. If it is a major revolution, naturally 
we would be led to assess the relevant cost of carry- 
ing out the changes. If it is realistically a minor 
change, then the cost considerations may not be 
all that relevant. 

Recent events at Berkeley arc good evidence 
of the uncertainty as to where wc are in educational 
development; the campus is more or less split on 
the issue of whether we should “reconstitute" our- 
selves. 

Philosophical Inquiry 

These remarks take me to the last of the re- 
search efforts which I've called the philosophical. 
In this style of inquiry, one goes far beyond the 
precise, well-controlled, quantified considerations; 
and consistency is certainly not the highest value. 
In a philosophical mode of inquiry, the purpose 
is to re\eal the underlying paradoxes of the intel- 
lectual process. The basic underlying paradox of 
all rcscanh on human societies can best be stated 
in the form of the following kind of question: How 
can we conduct systemic inquiry into the inqmring 
process? Tc say that it is possible to assess the 
costs and benefits associated with one type of edu- 
cational program compared to another is to assume 
that we have the capability of educating ourselves 
in such an ass'ssment. This assumption draws us 
into an immediate vicious circle. By what means 
can we educate ourselves about the best method 
of education witi'out having presupposed the very 
answer to the question that wc are asking? 

The paradox has many different forms. One 
can raise the ques’ion. for example, of whether 
all educational planning does violence to the human 
individual, because it is essentially involved in (ty- 
ing to figure out w*ys in which w* can change indi- 
viduals* lives without knowing if we have the moral 
right to devote our time and energy to such efforts. 
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Here, again, is another warning for the input- 
output enthusiasts. I by no means imply that inpul- 
output analysis should be abandoned; such an im- 
plication would put me out of work. But the ana- 
lytic enterprise will succeed nr better if it ex- 
amines its own foundations and, particularly, the 
morality of the enterprise 

Philosophical paradox is a difficult area of in- 
quiry. For individuals whose inquiring style is ex- 
perimental it turns out to be extremely frustrating; 
and their natural inclination of inquiry leads them 
to smudge the paradox, so to speak. Smudging 
paradoxes is not terribly difficult. The paradox it- 
self is based on very broad considerations that are 
not well substantiated. Paradoxes, for example, 
can be smudged by making very, very careful dis- 
tinctions concerning the meaning of terms. When 
one does make very careful distinctions, the para- 
dox often disappears in the area in which the dis- 
tinction is made. But the paradox by no means 
disappears in reality; because inevitably it will show* 
up in some other domain even more critically. I 
make these remarks about smudging paradoxes in 
the hope of avoiding a fruitless discussion about 
the relevant importance of conducting philosophi- 
cal research in the area of education. Such a dis- 
cussion is apt to be fruitless, because it is based 
primarily on differences in individual inquiring 
styles There’s nothing that can be said definitively 
to the experimental inquirer which would lead him 
to believe that philosophical inquiry has much im- 
portance. By the same token, there’s nothing very 
much one could say to the strongly philosophical 
or systemic inquirer that would lead him to believe 
that experimental inquiry is anything more than 
trivial. Perhaps if we can recognize that there are 
styles which, in large part, arise from our own 
intellectual backgrounds as well as our own per- 
sonalities, the discussion could take on a more 
general and fruitful emphasis. 

In summary, in this paper 1 have tried to go 
from a fairly specific question about educational 
policy and research to very general considerations. 
Within the more specific suggestions. I have tried 
to examine the ways in which research can be used 
to strengthen the teaching process. especially in the 
methodological disciplines. In the more general. 
I have tried to consider the very broad problems 
of educational policy and to contrast two different 
viewpoints: the viewpoint that the educational pro- 
cess at the college level is essentially a transmittal 
of knowledge versus the viewpoint that the educa- 
tional process is essentially the development and 
maturation of the inquiring style of the individual. 
With regard to the second more general effort, I 
well recognize that may individuals, following their 



own inquiring style, will be inclined to say that 
educational process should do both, i.e., should 
both transmit knowledge and develop individual 
proclivities for inquiry. To keep the paper from 
failing apart, I suppose it would be essential to say 
that any such synthesis based on the concept of 
“both" certainly raises the whole question in its 
most serious form. How can we accomplish both 
goals? 

Interest in Inquiry Killed 

I should say that anyone who responds to the 
second part of the paper by ‘‘both*’ has not experi- 
enced the educational process in the way my own 
individual mode of inquiry has experienced it. For 
me, the contrast is a forceful one in the area where 
I do most of my work, i.e., graduate education. My 
own mode of inquiry has led me to conclude that 
most of our M.A. and Ph.D. programs are sadly 
lacking in their ability to create a keen interest in 
continuing inquiry. It it is true that most Ph.D.’s 
do not do research after receiving their degrees, 
then surely the output of Ph.D. programs is poor. 
1 believe that most of our graduate programs, es- 
pecially in the social science area, simply tend to 
prcJuce people whose continued interest in inquiry 
has been killed off in the process of the program. 
So, although this has been in a sense a journey 
from the specific to the general, or from the prac- 
tical to the sublime or ridiculous, nevertheless, I 
think the end of the journey has some extremely 
critical implications with respect to the practice of 
education at both undergraduate and graduate 
levels. 

NOTE: As 1 remarked at the beginning, t well 
realize that many of the suggestions contained in 
this paper arc being carried out, in one form or 
another, in various kinds of educational programs 
of research. I am familia,' with some of them; many 
others 1 know nothing about. I’ve intentionally kept 
away from making references to the enormously 
growing literature in educational research, both at 
the experimental and the systemic and philosoph- 
ical levels, some of which I have borrowed from 
shamelessly. I find in my own reading that such 
rcfercncc-making is disruptive of the thinking pro- 
cess of the author; because one is taken from the 
author’s chain of reasoning into someone else’s in- 
quiring style, which never fits very well Repeated 
citations of the form. "As X says.” seem to be 
becoming an obnoxious method of defending one’s 
ideas. It is clear that a paper of this kind is. after 
all. a very personal document based on one man’s 
experience and his particular style of interpreting 
that experience. My debts are many and profound, 
but to include them in this paper would be a dis- 
service to the reader and to other writers 
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"f would not waste much t/me trying to devs/op an Index o t 
total knowledge, discovered or transmitted, In the hope that 
I could then use It (o evaluate alternative programs . . . Rather, 
1 would begin by trying to understand very well where we are 
now, and on what basla allocation decisions are now being 
made, and what might be done to Improve that basis." 

"A cost analysis may Identity some bad choices even with • 
out being able to Indicate the right ones. This point Is clearly 
related fo another equally Important one about program 
analysis; that Is, analysis should be conceived as the servant 
of Judgment, not a substitute for It." 

“ Universities need procedures tor systematically recall o- 
catlng resources to reflect changing priorities caused by 
such factors as changing patterns ot student demand. 
My Impression Is that most universities do not have such 
procedures.” 

"I would suggest... the greatly expanded use ot external 
examinations rather than examinations set for each course 
by each professor." 
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The purpose of this paper is to offer a few 
practical suggestions on the development and use 
of measures of the outputs of higher education. 
These suggestions are based mainly on my ex- 
perience in the development and use of the Plan- 
ning. Programming, and Budgeting System (PPBS) 
in the Department of Defense and on my experi- 
ence as a member of the Board of Directors of 
Georgetown University. They also reflect a con- 
tinuing interest in the economics of higher educa- 
tion and a modest amount of university teaching 
experience. They are preliminary and general 
thoughts, not definitive conclusions based on study 
in depth. Also, my experience in the Defense De- 
partment was in the use of data for analysis as a 
basis for program decisions; it was not in the tech- 
nical details of management information systems. 
And my knowledge of the state of management of 
higher education is that of an interested outsider, 
not of a professional in the field. 

Information Does Not Exit! In Vacuum 

Output and cost ink rmation does not exist in 
a vacuum. To be meaningful, each bit must be an 
answer to a precisely formulated question. In 
searching for output measures, it is important to 
keep the purpose of each measure dearly in mind. 
In the case of higher education programs, output 
measures mry serve several different purposes: 

1 ) Aids in allocation decisions within the uni- 
versity (fn this case, output measures 
might help answer a question such as, 
"Which programs or departmental budgets 
shall we cut. and by how- mu:h, to make 
room for a growing Computer Science 
Department?’) 

2) Aids in broader allocation decisions be- 
tween higher education and other public 
programs (For example. “What benefit will 
result for our state from the expenditure of 
an additional $10 million on higher edu- 
cation?'*) 

3 ) Aids in evaluating the effectiveness of af- 
ferent teachers, teaching methods, or cur- 
ricula (For example “Is computer-aided 
instruction better than the traditional meth- 
od of teaching elementary Russian to under- 
graduates?") 
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Measures for different purposes are answers to 
different questions. They do not have to be the 
same. In fact, they probably will not be the same. 
Moreover, we may have no explicit way of relat- 
ing one to another. 

Let me illustrate this with an example from 
program analysis work in the Defense Department. 
The Department has studied various programs for 
the purpose of limiting the numbeis of Americans 
who would be killed n the event of a nuclear war. 
The list includes: 

1) Air (anti-bomber) defense forces, includ- 
ing interceptor aircraft, surface-to-air mis- 
siles. radars, and communication and con- 
trol systems 

2) Anti-ballistic missile (ABM) systems 

3) Fallout shelters 

For purpose of allocation decisions among dif- 
ferent continental air defense systems, the number 
of enemy bombers shot down before they reach 
their targets is a good criterion. It is also a good 
cri.erion for evaluating the performance of the air 
defense system in exercises. But an analysis in- 
tended to aid such broader decisions as the alloca • 
<ion among air defense, anti-bal istic missile de- 
fense, and fallout shelters, or a decision on the al- 
location between these programs, taken together, 
and other programs must be based on a broader 
criterion, such as the number of lives saved in 
various circumstances. 

In this particular case — continental air and 
missile defense forces and other so-called "damage 
limiting programs" — we were able to develop a 
unified theory by which the specific performance 
indicators of individual weapon systems could be 
related explicitly to the broad overall objective of 
saving American lives. Thus, output measures use- 
ful for judging the balance between these and other 
programs were either the same as or could be 
explicitly related to output measures useful for 
judging the balance among these programs and for 
evaluating performance. But this was an excep- 
tion. not the rule. Only in a few cases was this 
possible. Generally it wasn’t. For example, we have 
r>o criteria or calculations that relate the speed or 
payload of cur fighter-bombers to our ability to 
defend Europe. And even in the case of the “dam- 
age-limiting" programs, the question of criterion 



52 



was not simple. The comparative effectiveness of 
different program mixes changed with different as- 
sumptions about the circumstances surrounding 
the outbreak and conduct of the war. Thus* value 
judgments had to be made about how much we 
wanted to protect in which circumstances. 

In this paper I propose to discuss mainly the 
first of these purposes of output measures, that is, 
aids to allocation decisions within the university. 
And I propose to limit that to undergraduate and 
postgraduate instruction. 

Developing Criteria 

In developing criteria for program decisions, 
it is important to set one's sights on realistic ob- 
jectives. It is difficult to go far beyond crude, sim- 
ple criteria; but such criteria can prove to be sur- 
prisingly useful. 

Development of output measures for public pro- 
grams is an inductive process, not a dcdcctive one. 
It is a matter ot cut and try, of successive approxi- 
mations. One should not expect to be able to de- 
duce a hierarchy of criteria that will logically order 
the university program. I doubt very much that it 
will ever be possible to do a satisfactory job of re- 
lating alternative instructional programs to such 
broad measures of social welfare as increases in 
state or national wealth, increases in knowledge, 
good citizenship, or personal happiness. Rather, 
one should start with simple, crude criteria, and 
then attempt to refine them. 

I learned this lesson the hard way in the De- 
fense Department. In the early 1960s, 1 began 
working on the question of tactical air force re- 
quirements. I started by thinking that it would be 
possible to develop a unified analysis for land forces 
and tactical air forces. We reasoned that the ef- 
fectiveness of the classical tactical air missions in 
a conventional war — air superiority, close air 
support, and interdiction — could be measured by 
their impact on the land force ratio between op- 
posing forces at the front, and thus *hat (he land- 
air '‘tradeoff' would be a decisive factor in de- 
termining the size of U. S. tactical air forces. Ap- 
proaching the problem in the manner of an econo- 
mist or operations analyst, my colleagues in the 
Systems Analysis Office and 1 tried to develop 
tradeoff curves for land and air forces yielding the 
same effectiveness. 

Unfortunately, we were quite unsuccessful. We 
simply could not find the relevant data with which 
to calculate how much better the land force ratio 
would be if we had another wing of tactical air 
forces in a particular theatre. Nor could we get a 



reasoned judgment from the military experts, based 
on the available data. 

After blunting our lance for several years on 
the land-air tradeoff problem, we realized that the 
actual decision-making was being based on much 
simpler criteria, such as a comparative count of 
enemy aircraft versus ours. And we realized that 
some military recommendations to the Secretary 
of Defense were based on counts that were badly 
wrong. They were wrong for several reasons. First, 
the definitions used had the effect of comparing a 
number close to the total inventory of our poten- 
tial enemies with a much smaller fraction of our 
inventory. Second, they ignored important qualita- 
tive differences between our and enemy airc-aft. 
For example, a typical NATO fighter-bomber air- 
craft can deliver about two and one-half times the 
payload of a typical Warsaw Pact plane over a 
representative combat sortie. 

So, in 1964, we switched our effort from try- 
ing to develop a sophisticated solution to the total 
tactical air requirements problem to just getting the 
numerical counts straight, and to developing ef- 
fectiveness indicators that would take account of 
the expensive qualitative advantages being built 
into U. S. aircraft. In retrospect, we should have 
made this switch sooner. 

One of our early efforts was a simple index 
of th* total payload that our forces and enemy 
forces could carry on representative sorties. This 
index showed that the total payload of U. S. forces, 
which was about equal to the total of all Com- 
munist forces in 1961. had more than doubted by 
1967, while the Communist total grew by only 
23 per cent. 

The Secretary of Defense used this index very 
effectively to refute charges that enemy forces were 
more powerful than our own. The use of the pay- 
load index was immediately met by the Joint Chiefs 
of Staff with the argument that "payload is not 
the same as force effectiveness." But the Secretary 
of Defense was able to respond along the follow- 
ing lines: # i agree. It's a very crude index. But 
it is a lot belter than just numbers of aircraft, and 
it gives a much more accurate picture of the growth 
in our total capability. So I am going to go on us- 
ing it until a belter measure is Available." And 
he did. 

Simple, Crude Indict* Helpful 

This experience taught me that one should not 
expect to find an all-embracing criterion of vahn 
added or effectiveness, and such criteria really 
aren't necessary for improved allocation decisions. 
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Simple, crude indices can be very useful. This and 
similar experiences inspired the motto, "It is bet- 
ter to be roughly right than exactly wrong.” 

Applying this lesson to the university scene, I 
would not waste much time trying to develop an 
index of total knowledge, discovered or transmitted, 
in the hope that I could then use it to evaluate 
alternative programs. That is a conceptual box that 
is very likely always to remain empty. Rather, I 
would begin by trying to understand very well 
where wc are now, and on what basis allocation 
decisions arc now being made, and what might be 
done to improve that basis. 

A related point is that analysis can illuminate 
program choices among quite dissimilar programs 
even in the absence of a criterion that measures 
them both. For example, implicitly, the Secretary 
of Defense had to allocate the defense budget be- 
tween continental air defense forces and airlift sea- 
lift forexs for the rapid deployment of our forces 
overseas. We had no unifying criterion spanning 
the output of these activities. But wc could calcu- 
late, for example, how many extra lives would be 
saved by an extra billion dollars spent on conti- 
nental air defense. And we could calculate how 
much faster a billion dollars would permit us to 
deploy reinforcements to key overseas areas. This 
information could and did illuminate such choices 
for the Secretary by making clearer for him the 
judgments he had to make. I am sure the same 
point applies in the analysis of alternative univer- 
sity programs. It isn't necessary to be able to 
measure the ultimate social value of alternative 
programs in order to improve the basis on which 
such choices are made, 

Universities aic thought to serve many pur- 
poses. Mark Blaug mentions a few; " To select the 
most able for leadership in industry and govern- 
ment. to cultivate talent for the sake of self-en- 
richment, to promote scholarship and scientific re- 
search, to preserve and disseminate cultural values 
. . . John Keller mentions several views: (1) the 
university's objectives are the preservation, trans- 
mission. and application of knowledge, and the aug- 
mentation of the stock of knowledge: (2) the uni- 
versity is a service industry' responsive to consumer 
demand; (3) the university is a producer of human 
capital for the economic development of society: 
(4) the university is a source of instructional, re- 
search, and public services. 1 The list could go on 
and on. It is clear to me that there never will be 
agreement on one or even a few of these purposes 
as preeminent or as the basis for allocation de- 
cisions. But such agreement is not necessary to 
the development of an analytical basis for sub- 
stantially improved allocation decisions. 



Because there is no agreement on purposes or 
on relatives values, there is r*o "optimum" program 
for the university. There are only better and worse 
programs. Avoiding bad programs is a sufficiently 
ambitious goal to keep us all occupied for many 
years. 

This is a very important point about policy 
analysis. Much of the literature on analytical meth- 
od is based on the notion of finding an "optimum” 
solution, given some unique criterion and a set of 
alternatives. But as a practical matter, there are 
many relevant criteria involved in every decision. 
However, finding a solution better by most or ai! 
of the relevant criteria than the one currently in 
operation, is often possible. So is the identifica- 
tion of programs that are quite clearly poor choices. 
Let me give a couple of examples. 

In the early 1960s the Secretary of Defense 
had to decide how many Minuteman ICBMs to 
include in the defense program. The Chief of Staff 
of the Air Force recommended some 2,400. To help 
the Secretary reach an informed judgment on the 
matter, we made a graph showing the number of 
targets destroyed as a function of the number of 
missiles for each of various assumptions. As one 
might expect, with increasing numbers of missiles, 
the curve became very flat, reflecting the fact that, 
after a point, missiles were being used to reattack 
for the second or third time targets that had al- 
ready been attacked once. In fact, the curves be- 
came pretty fiat around 1. 000 ICBMs. But there 
was no simple "point of diminishing returns" or 
"optimum point.” The curve didn't tell the Sec- 
retary what the right number of missiles was. But 
it did enable him to identify a lot of bad answers — 
that is. numbers of missiles substantially in excess 
of 1. 000 — and that was a very useful result. 

Stmllariliet Seen 

A great deal of tire useful Systems Analysis 
work in the Defense Department was the identifica- 
tion of such "flat of the curve" situations. 1 am sure 
that such situations can be found in most univer- 
sity programs. Two examples that come to mind 
(I believe but can't prove) are the fifth and subse- 
quent years in Ph D. programs and certain reduc- 
tions in class sue such as, for example, from 100 
to SO. When f have raised these issues on campuses, 
the replies have always been couched in terms of 
the absolute benefits of longer study and smaller 
classes. But that is the wrong way to look at the 
question The real issue concerns the marginal 
b:nefits — tbe extra know ledge associated with 
five years compared to four, o t classes of 80 com- 
pared to 100 — compared to the alternative uses 
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of the same resources. I don’t doubt that a man 
who spends five years on his Ph.D. is iikely to 
know more about his subject than he would if he 
had spent four, although whether lie will know 
more at the end of the fifth year than he would if 
he had spent the year some other way depends 
in part on how he would otherwise have spent the 
fifth year. But suppose the department in question 
has room for 100 Ph.D. candidates. If it takes five 
years for each candidate to get his degree, the de- 
partment will produce 20 Ph.D.’s per year. If 
the course can be reduced to four years, the output 
can be increased to 25 Ph.D.’s per year at the 
same cost to the university. The latter might be a 
much better use of university resources. 

At Georgetown, Professor Bruce Davie of the 
Economics department did an analysis of cost 
per student credit hour taught in the various de- 
partments and costs per credit hour enrolled in 
each of the various academic programs. The analy- 
sis showed that the cost per underdgraduatc stu- 
dent credit hour in one of the sciences was more 
than twice the cost in another. John Keller re- 
ported that a cost analysis of the Berkeley campus 
showed a six to one differential between the dis- 
ciplines with the highest and lowest costs per stu- 
dent year at a given level.* There is no way to prove 
that approximate equality of costs among disci- 
plines is right. But it isn’t necessary to obtain such a 
proof to get agreement that cost differentials be- 
yond certain limits are not acceptable unless ex- 
plained by special circumstances. Thus, a cost 
analysis may Identify some bad choices even with- 
out being able to indicate the right ones. 

This point is closely related to another equally 
important cnc about program analysis; that is. 
analysis should be conceived of as the servant of 
judgment, not a substitute for it. Thus, work on 
output measures should not be directed to an at- 
tempt to find criteria that will form a wholly ra- 
tional basis for decision-making, that will free the 
responsible officials from the burden of making 
judgments and hard choices. Such an attempt 
would be bound to fail. Rather, efforts on develop- 
ment of output measures should be made in recog- 
nition from the outset that dccision-mikers arc go- 
ing to have to make judgments. Thus, such efforts 
should start where the decision-makers are now, 
and attempt to increase the amount of useful in- 
formation available to them. 

Another lesson we learned from building the 
Planning-Programming-Rudgeiing System in the 
Pentagon is that there is no single “best” wav to 
slice the program data. Up to I960 the only way 
the Defense budget was subdivided was by Military 



Department (Army, Navy, Air Force) and by ob- 
ject class of expenditure (procurement, operations 
and maintenance, construction, research and devel- 
opment, and military personnel). This breakdown 
didn’t help to answer many fundamental questions 
about the shape of the Defense program, fn the 
early 1960s, under the leadership of Charles Hitch, 
then Comptroller of the Defense Department, we 
developed a mission or output oriented breakdown 
of the budget as the main frame of reference for 
program decision-making; that is. Strategic Retalia- 
tory Forces, Continental Air and Missile Defense 
Forces, etc. 

But we soon found that the budget data had to 
be sliced up in other ways also. For example, for 
studies of our posture in and for NATO, we had 
to develop a breakdow n by theatre. For analysis of 
traming. housing, medical support, etc., we had 
to look at manpo' ^r regardless of mission, etc. 

Similarly. I am sure that analysts of university 
programs will need to continue to break down the 
budgets by departnien*. school, degree program, 
level (lower division, upper division, etc.), object 
of expenditure, and other ways. Generally speak- 
ing. this means beginning with the budgets of or- 
ganizations that are cost centers, developing cost 
models that will permit estimation of other cost 
breakdowns, and it means building more flexibility 
into the management information system so that 
adaptation can be made to changing needs. 

Good Analysis Provides Ground Rules 

I remarked earlier that analysis is not a substi- 
tute for judgment. It is not a substitute for debate 
either. Rather, a good analysis can provide the 
ground rules for a constructive convergent debate 
that narrows differences and clarifies issues and 
points of disagreement. In fact, debate Is usually 
a necessary stimulant to analytical progress. It cer- 
tainly was the case during my years in the Pentagon 
that many of the most significant analytical ad- 
vances were developed in the midst of intense de- 
bate. And 1 am sure that a good source of output 
measures for academic programs would oe aca- 
demic departments that have been requested to 
justify their budget requests in such terms. Defense 
departments, universities, and other institutions do 
not discover their goals by speculating on them in 
abstract; they discover them by making hard choices 
from among the real alternatives they face. 

We had the beginnings of some useful debate 
last year at Georgetown University As I men- 
tioned earlier. Professor Bruce Davie of the Eoo- 
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nomics department did an analysis of cost per stu- 
dent credit hour. This exercise put the spotlight on 
some of the high-cost departments. Some of the 
high costs, of course, were justified. For example, 
it turned out that the Classics department was 
among the most expensive by this criterion. But the 
reason for this was that, although the department 
was small, the enrollment was proportionately 
smaller. To reduce the cost per student credit 
hour would have required reducing the size of the 
faculty. But agreement was easily obtained that the 
Classics department was already at a minimum 
critical size, so that proposals to reduce it would 
raise the question of its continued existence. But it 
was also agreed that it was an important part of our 
values at Georgetown to have a strong Classics 
department. Moreover, although the cost per stu- 
dent credit hour was high, the total cost of the de- 
partment was small. So no issue developed con- 
cerning this department. 

However, the story was different with other 
departments. The Mathematics department, as I 
recall, argued that a thrce-credlt course in math 
was equivalent to five-credit courses In other de- 
partments. That opened up such interesting ques- 
tions as the meaning of a credit hour — which 
forces one back to the question of output. 

As I mentioned earlier, Professor Davie found 
that one of the Science departments cost twice what 
another did to produce a student credit hour. Such 
differences can be analyzed and the reasons evalu- 
ated. Are faculty salaries higher in one department 
than another? If so. is it because salaries in general 
in one field are higher than the other? Or is it 
because one department head is a slrongei bargain- 
er than another? Or is there objective evidence 
that one department attracts a higher caliber of 
professor than the other? Or is it that one depart- 
ment contains more senior people than the ether? 
How much of the cost difference is explained by 
differences In class size and teaching loads? etc. 
Of cou 7 e. it can be dangerous to look at the cost 
side alone. A department could produce student 
credit hours very cheaply by resorting to very 
large classes and very low-pa*d personnel and by 
doing a poor job of teaching. It is also important 
to look at the quality of the product, seme indi- 
cation of which can be obtained through such 
means as graduate record exams. 

A process of analysis and debate in which 
high-cost departments are challenged to defend 
their position, ?f properly managed, can yield many 
useful insights. On the other hand, an analytical 
effort is unlikely to bear nu>ch fruit if it is ap- 
proached as a purely intellectual exercise of only 



academic interest. There are too many “unanswer- 
able questions 1 ’ that won't be attacked unless they 
have to be. My experience has been that, as a prac- 
tical matter, a necessary condition for good pro- 
gram analysis is that it is really tied in to the de- 
cision process. 

The fact that a cost-benefit analysis cannot 
prove the existence of a best answer should not 
be discouraging to the program analyst who starts 
with the right perspective. Choices have to be made, 
whether implicitly or explicitly. Rather than judg- 
ing his achievement against the model of the theo- 
retical literature in which the alternatives are de- 
fined and the criterion is unique, he should judge 
it against the actual situation in which decisions 
are being made with imperfect information and a 
multiplicity of criteria He should try to design or 
stimulate others to design new solutions that may 
be both more effective and cheaper. And short of 
that, he can identify programs whose costs are so 
out of proportion to their benefits that most dis- 
interested participants in the decision process will 
agree that action should be taken to change them. 

Incentives Needod To Produce Change 

Now* let us return to the purpose of output 
measurement. Suppose that we are successful in 
developing a set of meaningful and convincing out- 
put measures. Will this result in better program 
decisions? Not necessarily. One must look at in- 
stitutional decision processes and at the incen- 
tives that actually influence decision-makers. E\en 
with good information and good decision-makers, 
good decisions from an overall university or public 
point of view will not result automatically. 

Universities need procedures for systematically 
re-allocating resources to reflect changing priori- 
ties caused by such factors as changing patterns of 
student demand. My impression is that most uni- 
versities do not have such procedures. Of course, 
the process should be reasonably damped so that 
it will not over-react to temporary changes. And 
the process should allow for the application of 
value judgments. 1 would not wim to see the rigid 
application of a rule on cost per course wipe out 
the Classics department. But if student demand is 
growing in one are^, and declining in another, 
subject to these factors, re-cllocation ought to 
occur. 

We need to encourage experiments with dif- 
ferent class sizes. 1 personally believe that once a 
class exceeds 20 or 30 students, the teacher K 
for all practical purposes, lecturing (and he might 
as well be lecturing to 3 ,000| Classes ought to be 
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either very small, to permit the stimulation of 
dialogue, or very large, to take advantage of 
economies of scale. Classes between, say, 30 and 
300 are wasteful. Yet most university classes fall 
within that range, Many people do not agree with 
this view. And I have no evidence on the output 
side to support it. But evidence can and should be 
developed to test such hypotheses. If one is willing 
to accept the results of ‘‘before and after’* examina- 
tions as measures of value added — and I would 
be — then it should not be difficult to organize ex- 
periments that would provide answers. 

Experimentation with the use of technology, 
such as closed circuit TV and computers, ought to 
be expanded greatly. The lecture system in Ameri- 
can universities has long seemed to me to be ? 
poor use of resources. But if we must have it, why 
not do it with taped lectures on dosed circuit TV? 
This could solve the problem that many of our 
classrooms are not big enough for large lectures. 
The same lecture could be shown in several class- 
rooms. Also, by showing the lectures at different 
times, one could ease the difficult scheduling prob- 
lems universities face. Belter still, students in west- 
ern universities could get their lectures on Ameri- 
can Intellectual History from Arthur Schlesinger, 
Jr., and their lectures in Principles of Economics 
from Paul Samuelson, while their local professors 
would be freed to meet witk them in small groups. 
I think this last point would be the greatest ad- 
vantage. 

At Stanford, Professor Patrick Suppes and his 
associates have developed programs for computer- 
aided instruction. A program is now in use for 
teaching elementary Russian to Stanford under- 
graduates. Some students are taught in the tradi- 
tional manner, others with the computer program. 
Last year it was reported that the percentage of 
students successfully completing the first year 
course was substantially higher for the computer- 
aided group than for the group following the tra- 
ditional method. The main reason for this is that 
the student using the computer system can work 
at his own pace. 

It should not be difficult to combine such an 
experiment with a cost analysis to produce esti- 
mates, for example, of the cost to the university 
of producing a successful student under each 
method. 

No “Best” Solution 

This example can be used to illustrate a point 
that I made earlier to the effect that there is no 
“best” solution. One might think that cost to the 
university per successful student is a good criterion 



for choice between the two teaching methods. But 
it may not be an adequate criterion. Suppose, for 
example, that 60 per cent of the students complete 
the course under the traditional method while 80 
per cent do under the computer-aided method? 
Suppose, nevertheless, that the computer-aided 
method costs so much more that even the cost per 
successful student is 20 per cent higher using it? 
It is certainly not obvicus that one should there- 
fore reject the computer-aided method. One must 
consider the value of the students’ time. Presum- 
ably, those 20 per cent of the students who dropped 
out or failed under the traditional method, but who 
would have succeeded under the computer-aided 
method have pretty much wasted their time. And 
one must consider the value to those students of 
successfully completing the course. Value judg- 
ments on these matters are inescapable. And the 
values that are involved are of interest to the stu- 
dents, the university, the public (which supports 
the university), as well as the department pre- 
sumably making the decision. This makes the ques- 
tion of the incentives influencing the decision- 
makers all the more important. 

But many experiments and innovations that 
would improve effectiveness and efficiency are not 
going to be made as long as academic departmental 
incentives are the way they are. Academic depart- 
ments are not seeking to maximize output, given a 
certain budget, or to minimize the cost of doing 
their job. Rather, they are trying to maximize indi- 
cators of their prestige, such as numbers of mem- 
bers and of publications, and the freedom to pur- 
sue independent scholarly interests. My point is 
not that this is wrong, or that professors and their 
department heads are not good and public-spirited 
people. The point I want to make is simply that 
this is the way departments behave in fact. This 
is not a phenomenon that is unique to the academic 
profession. Ft has its counterparts in all walks of 
life. But it is a fact that we would do well to 
recognize explicitly if we want our efforts to im- 
prove university resource allocation to be fruitful. 

Let me give some examples. A number of years 
ago, at the end of a visit to a university economics 
department, I made a number of suggestions for 
the realignment of their teaching program. They 
were using the introductory courses as ‘Tillers” in 
the schedules of all their teachers, with the result 
that nobody had full-time responsibility for the in- 
troductory courses, the courses were not being 
taught well, and they were being taught very un- 
economically, With lectures being given to classes 
varying in size from 30 students to 300. I suggested 
to the department that it should select its most 
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effective and attractive lecturer, relieve him of all 
other duties, assign him the job of giving the intro- 
ductory lectures to very large audiences, perhaps 
at two different hours to facilitate scheduling. This 
would assure that the lectures would be done well, 
and it would save a lot of money because the un- 
economical lectures would be replaced by one or 
two very economical ones. Part of the savings 
could be used to pay for more small seminars 
and/or lighter teaching loads. Thus, it would be 
a very good thing from the university’s point of 
view, Also, it would be good from the point of 
view of the department’s own narrow interest. Ex- 
posing all the undergraduates to the department’s 
most attractive lecturer would help to increase the 
number of undergraduates majoring in Economics. 

My hosts agreed with the merits of the pro- 
posal, but said that they had no incentive to do it 
because the university would not let the department 
keep the savings. Rather, the effect would be a re- 
duced departmental budget. And so my proposal 
was not acted upon. 

Check the Catalogues 

You can find evidence of this problem by look- 
ing at the catalogue of course offerings of many de- 
partments in many universities. What you wiil often 
find is a crazy-quilt of specialized offerings reflect- 
ing the special interests of the professors, rather 
than a balanced, broad, and economical curricu- 
lum plan designed to take the student in an order- 
ly way through the fundamentals of his chosen 
field. 

“Why are you offering a course in the develop- 
ment of Adam Smith’s thought from 1774 to 
1776?” 

“Oh, that’s how we attracted Professor X away 
from Berkeley.” 

It is fine for the department’s prestige standing, 
but poor for the economics of the instructional pro- 
gram, 

The same incentives operate on departmental 
attitudes toward increased class sizes. Departments 
will resist increasing average clasu size from 100 
to, say, 120 (an increase that I believe must be 
“on the flat of the curve” fiom the point of view 
of teaching effectiveness) because that will weaken 
their cases for hiring more faculty. And I believe 
that even if computer-aided instruction can be 
shown to be both better and cheaper, it won’t be 
adopted by departments which see expenditure on 
computer time as an alternative to expenditure on 
staff. Given the choice between a red-blooded, 



paper-publishing, committee-attending, prestige- 
building colleague and a computer, what depart- 
ment head is going to pick the computer? Or use 
closed-circuit TV? Or redesign his curriculum? 

What can be done about the problem of de- 
partmental incentives? Recognizing its existence is 
a useful step, but there is no simple answer. Among 
other things, what any single university can accom- 
plish is limited by the fact that the national pro- 
fession in each discipline is an important factor 
governing curriculum and teaching method. And 
the incentives of the profession (e.g., “raise quali- 
ty”) may be different from and even opposed to 
the interests of the university and the public. 

Without preteMing that I have a solution to 
the problem, I would like to suggest three lines 
of attack on it. 

Three Lines of A’, tack 

The first is appropriate financial incentives, 
When a department proposes a cost-reducing inno- 
vation, it should be allowed to keep the savings, 
at least within limits and for a time. It is especially 
important that the budgetary pi >cess not reward 
the most inefficient with larger budgets. So often 
departments in any bureaucracy can successfully 
strengthen their case for bigger budget by doing 
their job less efficiently. Changes in budgets should 
be positively related by general rules to changes in 
numbers of students taught, or better measures of 
output if they are available. Changes should not 
be based on such things as complaints that a partic- 
ular^ faculty, is [overworked. After all, a department 
can always adjust the faculty workloads by various 
means, including design of course offerings. 

The second line of attack I would suggest is 
the greatly expanded use of external examinations 
rather than examinations set for each course by 
each professor. I think external examinations offer 
many advantages. First, they can set broad stand- 
ards of achievement, independent of the personal 
preferences of any particular teacher, and thus 
they can serve (with proper correction for the 
knowledge and ability of the student input) as 
measures of the teacher’s and department’s teach- 
ing performance. I should think it would be very 
helpful to a university administration to know how 
each of i*s departments is doing in preparing its 
students, compared to a large sample of univer- 
sities. Second, they can change the whole relation- 
ship between student and teacher. If the examiners 
are external, the teacher can become the student’s 
collaborator in the effort to achieve a common ob- 
jective, rather than being the student’s judge. 
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Third, externa! examinations designed to reflect 
a consensus of the profession as to what a student 
should know would give the teacher a strong in- 
centive to cover the broad fundamentals in his 
field, rather than spending al! of the student’s time 
exploring specialized byways. I think that VVICHE 
could make a very useful contribution to its mem- 
bers by taking the lead in organizing the develop- 
ment and use of such examinations. 

My third suggestion for attacking institutional 
incentives for inefficiency parallels one of the main 
McNamara reforms in the Pentagon, that is, the 
establishment of a strong office of program analysis 
and review — we called it the Systems Analysis 
Office — as an active participant in the decision 
process. Some universities already have established 
such offices with the title, Office of Institutional 
Research. But as I hope I have made clear in this 
paper, such an office should play a much more 
active role than that suggested by the title, “Insti- 
tutional Research/’ The office should do analyses 
and stimulate others to do counter-analyses. It 
should develop criteria for evaluating programs. 
It should seek out and evaluate new ideas, and tiy 
to put the good ones into application. For example, 
if an analysis shows that computer-aided instruc- 
tion is definitely better and cheaper for introduc- 
tory courses in modern languages, it should con- 
front the Modern Languages department with the 
analysis. If the department cannot disprove the 
analysis, the office should bring pressure to bear 



through the budget to get the department to take 
appropriate action. The office should challenge 
questionable programs in the budget review. It 
might begin by developing a model of the univer- 
sity’s costs that would permit estimates of cost per 
student credit hour and then cost per degree pro- 
gram, The latter could be used, for example, in 
analyses of the costs associated with long Ph.D, 
programs. From there the office might go on to an 
examination of the incentives the departments have 
to be efficient, and to develop ways of better align- 
ing the individual departmental interest with the 
overall university interest. And, in collaboration 
with the academic departments, the office might 
plan and manage a program of experimentation 
with different teaching methods. 

Good output measures are not likely to come 
from disinterested speculation; they are much more 
likely to emerge from the cut and thrust of debate 
over real program issues, provided that the decision 
process favors the side with the better analysis and 
criteria. 

NOTES 

1, “The Productivity of Universities," Minerva 6, 
Spring 1968. 

2, "Higher Education Objectives: Measures of Per- 
formance and Effectiveness," Management Information 
Systems: Their Development and Use in the /Administra- 
tion of Higher Education, Boulder, Colorado: WfCHE, 
October 1969. 

3, Ibid., p. 83, 
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"One of the great challenges to higher education today Is to 
continue to go about Its frequently disturbing mission without 
unduly disenchanting, antagonizing, and disrupting the pub- 
lic sector." 

" Nothing talks tike money, and we never get tired of listening 
to It." 

" ‘If we don’t watch out, tho degree may go the way of Confed- 
erate money’. This Is easy enough to write for someone who 
has already made It... but what about underprivileged blacks 
and other unskilled or semi-skilled underclasses? The degree 
fetish and other forms of credentialism are troublesome, but 
Is It possible to de-emphaslze credentials without de-empha- 
slzlng what the degrees or credentials are supposed to stand 
tor? The Ph.D. and other degrees and certificates could stand 
a good overhaul, but I hope the baby Is not thrown out with 
the stale bath water." 



"Students . . . seeking relevancy between knowledge and ac- 
tion, learning and life, Instead find ...research grantsman- 
shlp and academic arrogance. Faculty members make more 
and teach less; work shorter hours and take longer leaves; 
obtain rank from universities and pull rank on students; and 
iast but oot least , thoy seek tenure and avoid students." 

"No so clety can exist without a measure of order. Here you 
see ... my own special concern for law and order." 
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Higher Education and the Public Sector 

KENNETH S. TOLLE1T 



Introduction: Disenchantment with 
Higher Education 

I recently spoke to a group of high school 
seniors on the role of law in international rela- 
tions and said in the course of my speech that I 
was preoccupied with determining the require- 
ments for a universal community of concord and 
peace. A young student quickly commented that 
that was easy: All that was necessary was the 
spread of the Christian religion. I replied that I 
was not sanguine about his answer in any respect 
because my study of the history of religions, in- 
cluding Christianity, gave me little reason for be- 
lieving that religion was a source of the salva- 
tion of man on earth. He bitterly retorted that I 
was an example of what was wrong with higher 
education. As soon as you mad a few books and 
learn a little something, you lose faith in the power 
and force for good of religion. 

This smart young student s perception of the 
impact of higher education upon me and others 
expresses in simple form the anxious view the 
public sector has of the outputs of higher educa- 
tion. I use this example of the youthful suspicion 
that higher education seemingly desanctifies and 
confuses life not in order to suggest that educators 
should look upon the public sector as adolescent, 
but in order to clearly indicate the educator’s al- 
most absolute responsibility to take seriously the 
suspicions and queries of the public sector. There 
was a profound insight in the young student’s ob- 
servation which cannot be gainsaid. Education fre- 
quently does unsettle conventional faith and ac- 
cepted wisdom in such a way that society, natu- 
rally — just as the young student — becomes anx- 
ious about its consequences. Of course, this natural 
uneasiness about higher education has been ex- 
acerbated by recent student unrest, protest, and dis- 
ruptions. One of the great challenges to higher edu- 
cation today is to continue to go about its fre- 
quently disturbing mission without unduly disen- 
chanting, antagonizing, and disrupting the public 
sector. 

However, this is just one aspect of the higher 
education impact or output problem. It concerns 
higher education’s effect upon student belief and, 
thus, behavior. Yet, there are other impacts which 
compound the problem of public disenchantment. 
Higher educational institutions not only discover 



and transmit knowledge which undermines :acred 
beliefs, but, also, especially according to post-mod- 
ern youth and radical academic critics, create and 
disseminate science and technology which are pro- 
gressively and pervasively destroying the environ- 
ment and vitiating 4l tbe quality of life” — whatever 
that means. As if the undermining of beliefs, cor- 
ruption of behavior, and destruction of environ- 
ment were not enuugh, the public cost of and ex- 
penditures for higher education have risen at a 
staggering rate, particularly the federal outlay. 1 
This all adds up to what David Riesman calls 
44 . . . ihe collision course of higher education.” 
What is taking place is that, 

. . . tremendous pressures now on higher education to 
increase what it provides to diverse constituencies 
at the same time that the taxpaying public, as repre- 
sented in legislatures, grows somewhat disenchanted . 2 

State and local government outlays for higher 
education have increased substantially, also, so that 
the combined outlays of national, state, and local 
governments amount to about fifty per cent of the 
expenditures on higher education. 

Nothing talks like money and we never get tired 
of listening to it. The public sector’s expanded sup- 
port of higher education, which began with the 
federal Morrill Act of 1862 (the first College Land 
Grant Act), continued apace with state and local 
government involvement and interest in public 
education generally after the Civil War. One of the 
great contributions to United States development 
came from Reconstruction legislatures in the 
South which gave free public education its first 
real boost in the South. Although this financial sup- 
port aspect of higher education, together with the 
emergent predominant role in terms of number of 
students enrolled of public higher educational insti- 
tutions, more than legitimizes a serious attempt to 
view higher educational impact and outputs from 
the public viewpoint, one should be warned against 
overrating this legitimacy in principle. After all, 
about fifty per cent of the cash outlays needed for 
college attendance comes from students, their par- 
ents, private gifts, and other non-public or non- 
governmental sources. However, if one thinks in 
terms of economic cost , particularly “ . , . the loss 
of earnings to the student because he has chosen 
not to be in the labor force,” 3 then it is realized 
that the public is not paying the piper as much as 
it thinks or it appears. In fact, Dr. Bowen con- 
tends, 4, The student’s share at present is of the 
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order of three-fourths to seven-eighths of the total 
cost of education, including the educational ex- 
pense of institutions.” 4 This fact should chasten 
the overreacting interventionism of the public sec- 
tor into higher education and enhance the beseech- 
ing participation in the governance of higher educa- 
tion by students. Yet if higher education has great 
impact upon society in any way, then society, the 
public sector, should want to know what that im- 
pact is and evaluate, modify, or reinforce, as the 
situation may require, that impact. 

Impact of Hlghar Education Upon Society 

That higher education has great impact, one 
way or another upon society, few would deny. lust 
what that impact is, many would disagree. This 
state of consensus, disagreement, and confusion 
about higher education more than justifies this 
seminar on “The Outputs of Higher Education.” 

Before exploring the impact of higher educa- 
tion upon society, one other recent reaction to 
higher education must be dealt with briefly. I 
speak here of the re-emerging complaint that too 
many people want to pursue srme kind of post- 
high school formal education. Although it might 
be preferable to defer comment upon this elitist 
phenomenon until after I have explored higher 
education’s impact upon society, I think that I 
should do it nc.v, so that I can early reveal my 
strong personal bias that the expansion and ful- 
fillment of universal higher educational opportunity 
should be accelerated. I believe intellectual integ- 
rity requires the disclosure of biases or prejudices 
early in any discourse. 

A recent comment in Change* entitled, “The 
Involuntary Campus,” writes of too much emphasis 
on going to college and getting more and more 
advanced degrees. After noting that minority 
groups see the degree as a way of making it and 
employers demanding it, the article observes, “It 
all adds up to academic inflation.” It then attacks 
credentialism and concludes, “If we don’t watch 
out, the degree may go the way of Confederate 
money.” This is easy enough to write for someone 
who has already made it or is making it, but what 
about underprivileged blacks and other unskilled 
and semi-skilled underclasses? The degree fetish 
and other forms of credentialism are troublesome, 
but is it possible to de-emphasize credentials with- 
out de-emphashing what the credentials or degrees 
are supposed to stand for? The kh.D. and other 
degrees and certificates could stand a good over- 
haul, but I hope the baby Is not thrown out with 
the stale bath water. 

A similar, but more learned, criticism or com- 



plaint is made by Dr. Martin Trow in a recent 
Daedalus article entitled, “Reflections on the Tran- 
sition from Mass to Universal Higher Education."® 
Under subheading, “ ‘Compulsory’ Higher Educa- 
tion,” he observes, 

. , . the expansion of American higher education, 
in numbers and functions, is transforming it from a 
system of mass higher education into one that will 
bear responsibility for nearly all of the college-age 
population — that is, into a system of universal edu- 
cation.^ 

Although the trend toward a system of univer- 
sal higher education is obscured by the fact that 
upper-middle class high school graduates attend 
some form of post-seccndary education at a rate 
of over 80 per r*nt and the lower classes at about 
50 per cent, more significantly the lower classes 
and generally upward-mobile students perceive the 
high rate of middle class attendance as correlated 
to the middle class’ apparent well-being and status. 
Upper-middle class peer-group pressure may com- 
pel college attendance by many upper-middle class 
students, who are not motivated but bored, 
and thus may be disruptive and unappreciative of 
the opportunities higher education affords them. 
Tevertheless blacks and large numbers of other 
underclasses correctly, I believe, see post-high 
school education as a way to make it in a still 
very success- and middle class-oriented society. The 
amenities and perquisites of upper-middle class af- 
fluence may not be as satisfying as they appear 
from below , but tho:e below would like to rise 
above to decide for themselves whether affluence 
and high places are shallow, empty, or meaning- 
less. 

The Carnegie Commission 8 has taken a very 
progressive, but responsible, position on access to 
higher education. The Commission said: 

We do not believe that each young person should 
of necessity attend college. Quite the contrary. Many 
do not want and will not wish to attend, and it can- 
not be shown that all young persons will benefit suf- 
ficiently from attendance to iustifv their time and the 
exntnse Involved. We should resist efforts to create 
a ‘‘captive” audience in our colleges . , , We therefore 
oppose universal attendance as a goal of American 
higher education and believe that noncollege alterna- 
tives should be made more available and more attrac- 
tive to young people. 

We favor, on the other hand, universal access for 
those who want to enter Institutions of higher edu- 
cation, are able to make reasonable progress after 
enrollment, and can benefit from attendance.® 

This position is sufficiently progressive to in- 
spire widespread support, but it is not necessarily 
as progressive as it could be. There is much justi- 
fication in the Washington Post , March 6. 1970 
editorial comment upon the Commission’s distinc- 
tion between universal access and universal attend • 
ance> The Post said: 
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There is implicit in this distinction, we fear, ac 
acceptance of the idea that there are some who were 
born to be hev/ers of wood and drawers of water . . . 
But perhaps the ultimate goal should be to make 
everyone to want this (higher education) to incub 
cate in the country's youth, through its system of 
elementary and secondary education, a love of learn- 
ing and a capacity to pursue it. 

Although the Post may be implicitly demeaning 
the hewing of wood and the drawing of water, I 
believe it has set forth the necessary ideal. In the 
discussion to follow, I hope to set forth some fac- 
tual and further rational basis for my bias. 

With these preliminary and general comments 
out of the way, I can now turn to a more detailed 
discussion of higher education’s impact upon soci- 
ety from the viewpoint of the public sector. Var- 
ious aspects, processes, and products or outputs of 
education will be commented upon. They all inter- 
mesh in a complex kaleidoscope, I will look at the 
undergraduate, graduate, research, and social in- 
volvement or extension aspects of higher educa- 
tion. These aspects involve the teaching, researen, 
and community-service processes of higher educa- 
tion. These ?11 converge and have impact upon stu- 
dents, economy, politics, and culture. In order to 
give some coherence and organization to the dis- 
cussion I will approach these aspects, processes, 
and products or outputs in terms of society’s econ- 
omy, politics, and culture. 

Economic Impact 

Symptomatic of the underdeveloped stage of 
social science is the quick resort to economic con- 
cepts in order to explicate and rationalize social 
phenomena and institutions. This has taken place 
in higher education in part because of the self- 
consciousness of educators ai\d the public about 
the extent to which the public sector is subsidizing 
or financing higher education, It is also a compli- 
ment to economics in that it seems to have ad- 
vanced farther than most social sciences in develop- 
ing concepts for rigorous analysis of social pro- 
cesses. The very title of this seminar expresses the 
recognition that if one wants to measure what 
higher education is up to, it is useful to talk in 
terms of the outputs of higher education — outputs 
being, uniquely, an economic concept. 

Production Function 

Output is a concept in economics used to ana- 
lyze the production process. Labor, land, and ma- 
terial are inputs in the production process and 
cars, airplanes, food, services, and housing ac- 
commodations are outputs of the production pro- 
cess. Thus, one can speak of higher education, as 
Dr. Clark Kerr and Dr. Theodore W. Schultz 10 
have spoken of it, in terms of providing or per- 



forming a series of services related to production. 
This production function of higher education in- 
volves activities which add to the output of goods 
and services in society. In other words, “Higher 
education is engaged in three major types of pro- 
duction activities which entail discovering talent , 
uistntcllon, and research .” u To put this another 
way, higher educational institutions train people 
vocationally, they seek people with talent so that 
they can be edi cated and perfoim production func- 
tions for society, and they engage in research 
for the purpose of developing a technology and an 
economy which are continuously expanding. And, 
of course, the university through formal and in- 
formal consultations can render aid to the govern- 
ment and to industry. We know that there is an 
especially great concern about universities perform- 
ing this production function in a particular sphere 
and form — namely, giving advice, counsel, and re- 
search aid to the military-industrial complex. 

There is an inevitable ambiguity in speaking 
of the production function of higher education. 
Higher education as a production process or ac- 
tivity naturally implies it is subject to traditional 
and modern economical analysis. Since other in- 
stitutions engage in the same activities as higher 
education, immediately the question may arise how 
efficiently and effectively does it discover talent, 
instruct future workers, and do research compared 
to other institutions or to alternative ways of carry- 
ing out these activities? 

Furthermore, these activities result in two prin- 
cipal products or outputs: (I) Student-graduate, 
discovered and trained, and (2) knowledge. An- 
other question may then arise. What is the market 
for variously trained undergraduate, graduate, and 
professional students and for knowledge of var- 
ious subject matters? In other words, higher ed- 
ucation as a production activity must also be con- 
cerned about the problem of product mix and 
market. An obvious measure here is the demand 
for and success of various former students and 
graduates of higher education in obtaining remun- 
erative employment and satisfying careers. 

Yet the production function of higher education 
was operationally defined ns performing a series of 
services related to production. Although services 
may be regarded as and subjected to economic 
analysis just as well as goods, many services are 
not subjected to traditional or even new economic 
analysis. How do you really evaluate or measure 
the services rendered to individuals and societies 
by doctors, lawyers, social workers, and ministers? 
The more the services of these professionals have 
been deprofessionalized from the traditional sense 
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of profession and marketed in the traditional sense 
of business (such as setling soap or repairing tele- 
visions), the easier it is to subject them to conven- 
tional economic analysis. Inherent in the tradi- 
tional notion of profession is public and civic serv* 
ice, special learning, and high standards of ethi- 
cal conduct. Thus, it used to be more important 
for a doctor or lawyer to save a life than make 
a fee, for a minister to . . save a soul” than to 
be highly paid. Advertising or special merchandis- 
ing of the services of either in the past would have 
been regarded as unprofessional, vulgar, and cor- 
rupting. Although technical efficiency has always 
been important for the professions, economic ef- 
ficiency has not. Higher education as an activity 
which provides services, say, to production may in 
part be regarded as the traditional professions were 
conventionally regarded. If so, it would be inap- 
propriate to subject it to traditional or even mod- 
ern economical analysis. 

But alas, just as medicine and law are going 
more and more commercial and, thus, are inviting 
conventional economic analysis, management, and 
profiteering, so higher education may be going. 
Indeed, much of student unrest is a counter-revolu- 
tion to what Jencks and Riesman call the “aca- 
demic revolution .” 12 In discussing this problem in 
another context I have said: 

Students . . . seeking relevancy between knowl- 
edge and action, learning and life, instead find "ped- 
antry and alienated erudition." research grantsman- 
ship and academic arrogance. Faculty members make 
more and teach less; work shorter hours and take 
longer leaves; obtain rank from universities and pull 
rank on students; and last but not least, they seek 
tenure and avoid students . 13 

I suppose it is difficult for higher education, 
just as for other institutions and professions, to re- 
sist the inexorable forces of an urban-industrial 
society. To the extent higher education can avoid 
being just another economic entity it can invite 
standards of traditional, qualitative evaluation 
rather than economic quantification. 

Education Increates Productivity 

There seems to be much evidence that the in- 
struction students receive in higher education, 
both graduate and undergraduate, greatly increas- 
es their productivity as workers in the economy. 
Although economic studies may be engaging in the 
fallacy of confusing post hoc with propter hoc , 
all that is necessary to measure higher education 
productivity is to obtain statistics on the life-earn- 
ings of workers who have pursued post-high school 
education and those who have not. I think it is 
fair to say that the Identified greater income of 



those who have attended higher educational insti- 
tutions is correlated to their higher productivity 
than to those who have not attended higher educa- 
tional institutions . 11 

The talent hunt aspect of higher education has 
been woefully neglected. There is no question that 
higher education plays a great role in providing 
students with an opportunity to discover whether 
they have the particular capabilities that are re- 
quired for performing whatever work, vocation, 
or occupation they have chosen. I suspect that the 
greatest contribution higher education makes in 
the talent hunt aspect of our economy is that at- 
tendance and completion of post-high school edu- 
cational institution demonstrate a kind of self- 
discipline and self-determination which are val- 
uable assets in workers in almost any kind of 
economic enterprise. 

Related to this function of discovering talent 
in higher education is the problem of inadequacy 
of counseling, both vocational and psychological, 
in high school and post-high school institutions. 
Adequate counseling in high school should help a 
student tentatively, not only to determine for him- 
self, his career or vocational choice, but also, 
more importantly, to select the post-high school 
institution or environment in which his capacities 
and aspirations will be nurtured best. As a result of 
visiting a few highly selective and eiitist liberal 
arts colleges and talking to even more highly mo- 
tivated and skilled students, I am of the opinion 
that a number of post-modern youth are more 
interested in transforming colleges and universities 
into communes, encounter groups, and scenes for 
group therapy than in pursuing traditional higher 
education in its various forms of classical, scholar- 
ly. community and research-oriented institutions. 
This is not to malign these commendable searches 
for self-expression, improvization, sincerity, ident- 
ity, and authentic community. I just wonder 
whether higher education institutions are the prop- 
er environment for such pursuits. 

Although the research-service function of high- 
er education may sometimes be exaggerated, there 
is no question that the organized agricultural re- 
search of land-grant universities had a tremendous 
payoff in terms of the development of agricultural 
productivity and efficiency. The land-grant move- 
ment is one of the great success stories in higher 
education and you might say social legislation in 
the history of the United States. Unfortunately, no 
adequate research has been done or studies made 
of the actual impact of organized university re- 
search on other aspects of the economy. One can 
safely assume that it has had some substantial posi- 




tive impact, but it is not altogether clear whether 
a comparable impact could have been derived more 
economically from a greater expansion of research 
and development by private corp nations or special 
research institutes like the Rand Corporation. Cor- 
porations like Rand may be on the wave of the 
future. Certainly, many have already developed; 
if universities continue to be agitated by confronta- 
tions and disruptions, many more research scholars 
will repair to these research institutes. Research 
apparently has been separated from talent hunt 
and training in Russia for some time without any 
noticeable adverse effects upon research. 

En;laves Are Undesirable 

This last observation leads me to nnke a com- 
ment on the research activities of tht, universities 
which could more logically be left for my discus- 
sion of politics and culture. Here, however, logic 
may do violence to common sense. A recent arti- 
cle by Caryl A Haskins in The American Scientist, 
entitled, “The Humanities and the Natural Sci- 
ences, suggests that there may be grave risks 
run by society or the public sector if research or 
investigational enclaves are separated from uni- 
versities and thus separated from meaningful and 
vital association with the humanities and soft social 
sciences. After surveying the many fantastic ad- 
vances in science, particularly the probability that 
genetic change can be controlled, Haskins asks for 
what purpose or toward what end. He answers this 
question with the observation, “It is the humanist, 
it seems to me, who must be prepaied to provide 
the vital contributions here.” 16 

Suiprisingly, many of the advances in science 
which Haskins speaks about took place in what he 
called investigational enclaves t an environ- 

ment shielded, at least in good part, from the 
windier currents of the world, the inherent right 
and duty of those comprising it to preserve a work- 
ing climate effectively, and where necessary vigor- 
ously, protected against demands of the wider 
world for immediate and short range return.’ 117 
Haskins believes that the humanities and I sup- 
pose, also, the social sciences are especially con- 
cerned with the doctrine of the final worth and 
sacredness of the individual and the doctrine of the 
extraordinary importance of the life of the mind 
in our society. 

The matter of an investigational enclave is es- 
pecially important. Research institutes can partially 
meet this need, but I believe, for the humanizing 
purposes Haskins speaks of, it will be necessary 
to insure that a significant number of these enclaves 
stay at the great universities. For example, the head 



of the Economics Department of the Rand Corpo- 
ration recently led a discussion at the Center for 
the Study of Democratic Institutions on the mill* 
t ary -industrial complex. It was one of the most 
frightening discussions I have ever heard. 

\ Simply put, he tried to suggest that all >r, at 
least, most of the criticisms made of the military- 
industrial complex in terms of secrecy, complexity, 
institutional rigidity, scale, et cetera, could also be 
made of the health-educational complex. The kind 
of rigorous technical analysis he brought to bear 
on the military-industrial and health-educational 
complexities, as he called them, can be useful — 
especially as an intellectual and chastening exer- 
cise. However, when he, with self-satisfaction in 
his oral presentation, indicated that in terms of un- 
inflated dollars the military budget had increased 
only ten billion dollars between 1961 and 1971, 
while the health and education budget increased 
from 2 to 22 billion dollars in the same period, I 
almost blew my top. I am afraid that many of the 
criticisms made of Rand are justified. These crucial 
matters seem nothing but a numbers game to them. 
They talk about allocation of resources for warfare 
and killing as if they were indistinguishable from or 
comparable to the allocation of resources for health 
and education. The humanities do have an impor- 
tant role to play, which they can only play in rela- 
tion to scientific and technical research, if they 
are both part of a common community such as a 
university. 

Much more could be said about the contribu- 
tions research activities at universities make to our 
economy and society. I think this matter of re- 
search institutes being isolated from society and 
economic analysis, abstracting and computerizing 
with numbers and digits vital social processes, leads 
to a discuss ; on of the importance of the political 
contribution of higher educational institutions to 
society. 

Students React to Technology 

However, before leaving a discussion of the 
economic impact of higher education, a word or 
two should be said about students 1 reactions to 
economizing and computerizing life and about the 
consumption function of education. Michel Crozier, 
in a brilliant article 18 describing his impression of 
America upon revisiting it, noted great changes 
and the fact that, with the exception of Spain of 
all Western nations, America has had the most 
bloody past. He observed about the Eisenhower 
years, “It was natural for America to be conserva- 
tive, to have lost confidence In its creative ability, 
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to seek refuge in a ponderous moral complacency, 
showing beneath its hypocrisy an inferiority com- 
plex toward communism and even toward Europe.” 
This complacent and conservative prosperity was 
followed by a period of “ . . . aggressive confidence 
in human reason and the capacity of America to 
solve all problems by its use,” PPBSs, computers, 
and other instruments of technocrats inspired an 
“ . . . arrogant pride in the powers of rea- 
son” which both blacks and post-modern students 
were reacting to, although for different reasons. 
The former are in revolt, according to Crozier, be- 
cause they have found their civil rights gains value- 
less since they can be capitalized upon “ . . . only 
through the ability to play the game of modern 
calculation .... The more rational the society 
becomes, the more he loses his foothold. His eman- 
cipation is of no use to him, because he is much 
less competitive in today’s soefety than he was in 
the traditional industrial society.” I suppose one 
might infer from this analysis that this situation 
and predicament encourage blacks to engage in 
tactics of bombast, militancy, and hyperbole. He 
concludes about blacks that the apocalyptic aura 
should not hide the basic fact that ”... their 
problem stems from the intellectual change, not 
from a will to seize power, ” 

On the other hand, students are in revolt be- 
cause they are fascinated with the system and its 
rationale but, yet, they find no respite or secret 
corner hidden from the technological juggernaut. 
Thus, students are ”... dedicated to the cult of 
passivity and drugs, to living in happy abandon, to 
the refusal to shoulder the burden of calculation 
and responsibility.” There emerges and endures a 
conflict between ”... the rational and the spon- 
taneous, the community and the individual, the 
desire for freedom and the fear of responsibility.” 
Crozier’s answer or advice is: 

Perhaps, in order to carry the debate further, 
Americans will have to abandon this insoluble riddle 
and re-examine, through (heir institutions, the mini' 
mal degree of anarchy, confusion, and inefficiency 
which must be tolerated if the participants are to 
withstand the rigors of the new rationality. 

Although higher educational environments at 
many schools reinforce or even encourage such a 
reaction to reason, I believe this reaction is as 
much a product of student inputs — talents, skills, 
aspirations, affluent backgrounds — as it is the 
impact of higher education upon students. Post- 
modern white youth are in the midst of psycho- 
cultural revolution and are not preoccupied with 
“making it.” Indeed, they are not sure “making 
it” is worth the effort. 

Crozier's analysis of the malaise of blacks has 
enough validity to give it some credibility. How- 



ever, appeals to catastrophe are inspired more by 
knowledge and experience of unspeakable atrocities 
visited upon blacks than by insecurity and uneasi- 
ness with calculation, responsibility, and techno- 
industrial rationality. Aft^r all, the Kerner Report 
emphasized the overriding importance of racism in 
explaining the predicament of blacks in America, 
and the authors of that report suffer neither from 
an inability to play the game of calculation nor 
from paranoia or apocalyptic obsessions. 

The consumption function of higher education 
is related to the activities of students and the cam- 
pus community which involve consumption of 
goods and services of the economy or to the devel- 
opment of tastes, sensitivities, and opportunities 
which lead to life styles of consumption by stu- 
dents on a long range basis. This is closely related 
to the cultural impact of higher education upon 
students. As it will be briefly indicated later, this 
function cf higher education is in competition with 
mass media, particularly television. 

Political Impact 

The political impact of higher education may 
be approached by way of Dr. Kerr’s notion of the 
citizenship function of higher education . 19 Now 
the citizenship function is a function that’s very 
important. Here we are talking about thoao ac- 
tivities which 44 . , . relate to the performance of 
students, alumni, and faculty members in rela- 
tion to civic responsibilities.” It is carried out in 
a number of ways. One way, I suppose rejected 
by many, is completing students’ socialization. This 
is resisted by some and criticized by many. Cer- 
tainly it is criticized by radical students and aca- 
demicians. They feel that universities are too much 
involved in the process of socialization, that they 
arc trying to train students to accept society as it 
is, und that the universities are status quo oriented 
and implicated. 

However, I think it is sometimes forgotten in 
criticizing the socialization function which higher 
education sometimes contributes to, that no so- 
ciety can exist without a measure of order. Here 
you see creeping through my own special concern 
for law and order, 

The public sector should be especially inter- 
ested in citizenship. If educational institutions in 
our democratic society do not teach students about 
the American heritage of freedom (I have a little 
trouble saying, “heritage of freedom”; I haven’t 
had it, bi many whites have), then how will 
citizens develop a deep understanding and appre- 
ciation of it. I see nothing wrong with educational 
institutions trying to communicate to the r student 
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clientele a heritage of freedom and to teach some- 
thing about the constitution, something about what 
democracy means, something about the complexity 
of social order and the fact that there is a need for 
people to make some adjustment to each other 
and even to society. Although socialisation may 
be contrary to an ideology of self-expression and 
spontaneity, if every individual were to pursue his 
own thing, oblivious of other people’s things, so- 
ciety would run into intolerable conflict and chaos. 
This is not to <ay that one cannot do h.j own 
thing. I believe very much in one’s doing one’s 
own thing, but within a framework; of social order. 

PrmnrtUcrt ol Social Damocraey 

Professoi Jill Conway, in a recent article en- 
titled. "Styles of Academic Culture,” 50 suggests 
that the main social function of education, particu- 
larly universal education, is " . . . the preservation 
of social democracy” in \mcrica. She observes 
that John Dewey diagnosed the problem of public 
education as not having ”... functioned as an 
agent of democracy in America, because it had 
taught the literary and aesthetic values of a Euro- 
pean aristocracy.”* 1 The public education cur- 
riculum was redesigned In response to this criti- 
cism and the land-grant movement in state higher 
education bad already inaugurated vocational train- 
ing and community-service activities. This political 
«* sKial function of higher education I, as created 
the expectations and circumstances which have 
given rise to some of the disenchantment with 
higher education already mentioned. Professor 
Conway says. 

Detain* of the mythk dimensiom of the school 
t< ctntodttft of the democretk future, the cdoca- 
IkvuI community is automatkaliy perceived in 
Amerkr a« the community that can resolve alt social 
problem i 

Professor George Kateb. in a useful but curi- 
ously critical revi'w of scvcial books on educa- 
tion. put the same matter as follows : 

Mow can the academy endure the s.rain of being 
mothet and fa*her. leader and vktim. asylum and 
enemy. play yrosmd and bospi'd- bealer and ciiminat. 
reformer arid delinquent, church and state? Every- 
thin# <s expected of it. nothing is forgiven it.** 

I believe these high expectations regarding 
higher edircttion are the results of its former suc- 
cesses. particularly its performance in the land- 
grant university development. Yet the universities 
seem to be failing miserably in their socialisation 
function, if socialisation is defined merely as ad- 
justment to the existential social o*der. This 
naturally leads to a discussion of the cri.nal evalu- 
ation service of the citirenship function of higher 



education. However, before turning to this serv- 
ice or output of higher education, I should like to 
refer to an article which mixes political and eco- 
nomic considerations in appraising higher educa- 
tion. 

Hlphar Education a» a Form of 
Industrial Approntlcaahlp 

Christopher Lasch and Eugene Genovese in a 
recent article entitled, "The Education And the Uni- 
versity We Need Now,’’” state that neocapitalism 
must exclude large groups from production because 
they a v ' economically superfluous — they must 
be kept in places of detention. Blacks, the new poor, 
young people, and women constitute the large 
groups which need to be excluded, according to 
Lasch and Genovese. This crypto-Marxist politi- 
cal-economical analysis is worth further elaborating 
because it is strewn with evocative insights and 
elitist humanism. 

They continue their an&lysls by Ironically ob- 
serving: 

... (he gradual achievement of universal educa- 
tion. tike many other reforms that appear now only 
to have hastened the coming of the ••fechnotronic to- 
ckly," was \» rested from the rutin# class in the face 
of determ ned opposition. 11 

Furthermore, the tendency of education since 
the seventeenth century has been to make schools 
into total educational environments whereas the 
medieval concept of education viewed the pupil 
as fp.-c of supervision outside school hours. In loco 
porenth developed with residen'ial colleges which 
supervised all aspects of students’ lives. Young peo- 
ple became segregated in a prolonged slate of 
adolescence — a state of dependence and subordi- 
nation. Young people of the twentieth century have 
been glorified in a mann r similar to the glorifka 
lion of womanhood In the nineteenth century in 
order, cynically to keep women in a subordinate 
position. ”... but whkh many women Internalised, 
just as many young people internalise the glorifica- 
tion of youtli and remain permanently adolescents, 
emotionally, intellectually, and — not least — po- 
litically.'’ Bourgeois society can solve problems of 
youth no better than other problems with which it 
is faced. Neocapita’ism cannot solve them without 
committing suicide, say Lasch and Oenovese. to: 

. . . destroy ib* custodial function of schcolt: 
dissociate education front the process of pterion# 
qoatifkatioris fo» work, so far as lies b rvcsiNe, 
and where it k not. recognise mote frankly the char- 
acter of education as apprenticeship ahiVe seekin# 
to improve apprenticeship itself; and finally, provid- 
in# acceptable alternatives to formal schooling. both 
foe young people and -*■ equally impomfti — for 
adults. 
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They suggest a partial solution Is to shift tech* 
nical training to secondary schools and, thus, re* 
tease the university from its custodiat responsibili- 
ties. Of course, I would observe that the custodial 
responsibility logically is urelated to the matter 
of the nature or place of schooling, for that matter 
schooling or training at all. 

They furthet support the earlier stated notion 
that there is no correlation between education and 
Industrial efficiency. Credentials are more impor- 
tant in getting jobs than in actually performing 
them. Yet, somewhat inconsistently, they argue 
that the most striking function of higher education 
apart from the custodial function is training an 
army cf intellectual workers on which the cor- 
porate system depends. "Higher education has fee- 
come another form of industrial apprenticeship." 3 * 

This makes higher education "... another 
form of production.” Thus the ruling class wants 
higher education, like secondary education, o: 

1 ) train competent intellectual workers, 

2) find compensation for powerlessness >n a 
culture of consumption, and 

3 ) mind their own business in matters of state. 

In short, Lasch and Genovese see higher edu- 
cation as producing intellectual proletarians. 

The first comment 1 should like to make upon 
the above analysis is that it appears to criticize 
higher education for performing too well the serv- 
ice of discovering talent and tialnlng workers for 
the economy. From an economic viewpoint Dr. 
Schultz would flatly challenge the suggestion that 
it Is wrong for higher education to sene society 
and the economy the way secondary education has 
served them, in fact, he indicates that the private 
rate of return on secondary education when educa- 
tion is viewed as an investment in human capital 
is nearly twice that of college graduates.** The non- 
pecuniary returns are more difficult to calculate. 

Second, when assessing ron-pecuniary returns 
of higher education, there is much evidc.-ice that 
higher education is falling miserably, at kast at 
the most prestigious and so-called distinguished 
universities, in compensating students for their 
powerlessness in a culture of consumption and in 
inducing studen'.s to mind their own business in 
matters of state. 1 had thought students have been 
in the vanguard of the Civil Rights, the Reace. 
and row the Ecology Movements. True, a fringe 
of post-modern youth, ironically called the coun- 
ter-culture movement in its effort to avoid co-opta- 
tion by a technocracy of hyper-consumerism, has 
opted foe a flight from politics to poetry and from 



rational persuasion to spiritual conversion.” This 
may be a good place to turn to a discussion of the 
critical-evaluation service or aspect, process, and 
output of the citizenship function of higher educa- 
tion. 

Preparation for Social, Political Conflict 

The critical-evaluation aspect of the citizenship 
function of a university results In a tension be- 
tween this aspect and the socialization aspect. Here 
the university must carry out an operation similar 
to that which Roscoc Round spoke of regarding 
taw; that is, it must nurture stability and yet not 
stand still. 

A most eloquent and responsible spokesman 
for (he critical and evaluation function of higher 
education is Robert M. Hutchins, former Presi- 
dent of the University of Chicago.** Although he 
was a revolutionary innovator in higher education, 
he did not think society would toterate a revolu- 
tionary educational system; still he thought one 
of man's missions on earth was to change his en- 
vironment which is inconsistent to a large extent 
with the socializing asy-.-ct or process of higher 
education. Thus. I should suppose he would say 
that (he proper concept of a university is that it 
is an Intellectual community and an independent 
center for the pursuit of truth. This will require 
critical discussion, questioning, and debate of every- 
thing from the microbe and the molecule, to man 
and society, to nations and (he world, and to the 
world and the universe. 

Many modem radicals would regard Presi- 
dent Hutchins' critical discussion, questioning, and 
debate innocuous, gutless, and escapist — if not an 
elegant cop-out. Criticism and evaluation, to them. 
ai e only important when they reach the level of 
agitated dissent which calls for direct action and 
Involvement in society. The direct action and in- 
volvement must be comparable to the intensity of 
their Indignation and outrage and which 4*e stimu- 
lated by their unshakable fudgment and percep- 
tion of the present social order as being irredeem- 
ably corrupt and violent. Their critical and. in 
many respects, sound perception of. say. the United 
States as a hypocritical, callous, racist, and violent 
society leads them to question its legitimacy. Re- 
pressive over-reactions to their confiontations, 
demonstrations, and protests have contributed to a 
widespread conviction among many of them that, 
to quote Friedenberg. ' They are oppressed by a 
fundamentally illegitimate authority.”** They react 
to their perception of the illegitimacy of their re- 
pressors somewhat in the way Dostoevski's hero 
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reacted to the presumption of God's non-existence: 
Then everything is permitted. 

SludtnU Rt|tci Hypocrisy 

With all of my deep sympathy for the post- 
modern youth's emphasis upon style . action, and 
process rather than ideology; upon the ore hbility 
,vv?/> with its dishonesty and its separation of prin- 
ciple and action; and upon their fixation upon 
violence, with them rejecting and projecting and 
electing it — I do not and cannot accept the 
proposition that perception of illegitimacy or any 
other evil permits any and all types of action 
against it. What they have done is reject hypocrisy 
and replace it with their own elitist self-righteous- 
ness. 

Now Nathan Gfarer 70 has suggested three pos- 
sible political stances a university can take vis-a-vis 
society. One. that it can educate its students and 
pieparc them to act in society. Two. that it can en- 
tourage students to participate actively in the re- 
form of society. Three, that it can engage Itself 
in the radical reform of society by becoming a 
revolutionary university or partisan camp. I think 
It's obvious that if a university becomes revolu- 
tionary it will not survive. Certainly, the public 
sector has no obligation to support or encourage 
the existence of a revolutionary university. 

I hope the above discussion will not be in- 
terpreted as meaning I believe that higher educa- 
tion. through its students, research, community in- 
volvement or other activities, should not make 
waves. Quite the contrary. Yet. it should not be 
insensitive to the sensibilities and concerns of tl*c 
public sector. Perhaps it would be helpful if t 
interjected here my personal concept of educa- 
tion. 1 believe education is essentially aw intel- 
lectual process whkh requires the imaginative and 
creative development of the mind, the continual 
enrichment of human sensitivity and sensibility, 
and die purposeful pursuit of an understanding of 
the relationships between man and man and man 
and nature. Obviously this conception of educa- 
tion would constrain me to believe that higher edu- 
cation not only should prepare students for politi- 
cal and social activity and involvement but also 
should encourage such activity and involvement. 

I like $. E. and Zelfa LuriaV 1 discussion of the 
role of the university The authors pose three key 
alternatives for a university: 

I. Accept predominant values of and identify 
with society. 



2. Reject society's values and withdraw into 
traditional ivory tower. 

3, Develop a critical creative relationship with 
society. 

They opt for the third alternative. ! would add 
that there should be enough leeway and pluralism 
in higher education to tolerate or permit different 
universities to opt for any one of the three alter- 
natives. This may appear muddlchcadcd. 1 sup- 
pose it is. I would add that I think an undergradu- 
ate school would serve society better if it chose 
the third alternative; a graduate school, in some 
respects, could better pursue the second alternative; 
and university extension services or social involve- 
ment logically ould pursue the first alternative. 
The research activities of university could pursue 
all. any one, or any combination of the alternatives, 
f certainly agree with the Lurias when they said, 
“In our society, the university may be the most 
effective structure thiough which Intellectual forces 
cm be put to use In Influencing the course of 
social evolution in a rational way." 12 This may be 
as good a place as any to turn to a discussion of 
the cultural impact of higher education. 

Cultural Impact 

Thus far we have been discussing higher edu- 
cational impacts upon aspects of the social order 
which are peculiarly congenial to a public sector 
viewpoint and appraisal, although the appraisal 
cannot be made with scientific rigor. If for no 
other reason than that it is paying a large part of 
the bill of higher education, the public sector is 
in ercstcd in a certain measure of economy or eco- 
nomic efficiency in the various aspects, processes, 
and products or outputs of higher education Yet, 
historically, the motive force of public higher edu- 
cation has been as politically determined as cro- 
nomtcally determined — preservation of social de- 
mocracy. Of course, the pragmatic spirit of Ameri- 
cans quickly recognised the economic benefits of 
a well-trained labor force and of technological 
pttgress supported or stimulated by university- 
based research and extension or community serv- 
ices. 

The public considering in tandem the eco- 
nomic and political benefits of higher education 
has served well, what Dr. Trow calls, the popular 
functions of higher education. " . . , those activi- 
ties and purposes that the university . . . takes on 
in response to external needs and demands."” ft 
is very fortunate for higher education and America 
that, initially, private education dominated higher 
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education and afforded it an opportunity to de- 
velop a tradition of intrinsic functions, or at least 
functions which were not dependent upon any ex- 
ternally imposed popular consensus which could 
easily have been loo short-range in its view. That 
the populist land-grant college movement WbS so 
successful does not detract from the value and 
importance of what Dr. Trow calls the "autono- 
mous functions" of higher education, ", . . activities 
and purposes that the university defines for itself" 
Indeed, too strict an economic analysis of higher 
education either internally or externally would 
probably be very unwise for society in the long 
run and disastrous for higher education in both 
(he short and long run. Although education has 
been a most powerful and effective servant of eco- 
nomic processes, it is not essentially an economic 
process itself. Unfortunately, economic progress 
like technological development lends to swallow up 
the ends and values it is supposed to serve. Thus, 
not only are the time, land, capital, and labor 
devoted to education regarded strictly and almost 
solely as commodities to be measuied i t dollars 
and valued in an economic caiculus. but aUo arc 
the educational process and products so regarded. 

Dominant Thrust 

This latter thinking, which seems to be the 
dominant thrust of educational polk? in the Nixon 
administration.*' mav lead to a kind of educational 
open market and entrepreneurship which would be 
disastrous for the intrinsic and. I believe, cultural 
values of higher education. Of course, the policy 
of so open educational market ; s supposedly hu- 
manized or democratized by an elaborate and ex- 
pansive educational loan program, thus, all pros- 
pective students would have ct be loaned enough 
money to attend college, and the various univcrsl- 
tks and colleges would comr/ete like soap, clothes, 
and food ptxxlueers for their patronage. Madison 
Avenue would then have to take over the mer- 
chandising and packaging if not the processing of 
higher education. 

Michael Clurman in a recent article** con- 
demns the dual price >ystem of public and private 
education and the subsidization tf institutions: 
however, he lauds the market and its functions 
and urges subjectizn of higher education to the 
price system by charging full-cost tuition fee with 
government providing modest grants, liberal loans, 
and a universal repayment system through taxes. 
This is bis pos.'ion, although earlier in the article 
he said tbat higher education's financial problem 
was not gene- al poverty as a whole, but "... the 



intense competition for academic prestige and dis- 
tinction among the various colleges and universities 
which accounts for the financial insecurity experi- 
enced by so many institutions of higher learning.” 4 * 
In other words, the cure for the *lls caused by in- 
tense competition is more competition under the 
benevolent aegis of the invisible hand of the price 
system. 

Surely, the cultural Impact of higher education 
has been very great upon its outputs. This is so 
not so much in terms of higher education transmit- 
ting high culture which traditionally has been one 
of its autonomous functions but in terms of cul- 
tivating "... aesthetic sensibilities, broad human 
sympathies, and the capacity for critical and inde- 
pendent judgment." ,, This fact goes a long way In 
explaining the radical discontent of so many of 
the good students at not only the so-called best 
educational institutions, but also the less famous 
and prestigious institutions. What has happened is 
that this generation of college students, less pre- 
occupied with personal welfare or vocational suc- 
cess than prior generations, secs, the great poten- 
tial for the realization of the American Dream for 
alt and yet finds the country wasting its youth and 
resources in a violent and obscen; assault upon 
the land and people of Southeast Asia; spending 
billions to send men to the Moon, but balking at 
thv expenditure of a few million dollars to clear 
ghettos of rats: destroying Its environment; refus- 
ing to do justice to blacks and other underpriv- 
ileged groups. As l have said before, "It is in large 
measure because universities and colleges are sup- 
plying youth with a superior education (hat they 
perceive more clearly than past generations the 
stalk contradiction between what is preached and 
what is practiced . . . "** by this country. If the 
economic and political institutions of this country 
do not become more responsive to many of the 
grievances the post-modern youth have dramatized, 
then it is on a collision course with disaster. 



Mori Censdou* Than Betora 

This is so because the middle-aged generation, 
by virtue of its reasonably good education, is more 
conscious than earlier generations of problems and 
society's capabilities to cure (hem. It is just a 
little less sensitive and more tolerant of problems 
than post-modern youth. Furthermore, in part, 
because of the piorification of youth already re- 
ferred to. adults now learn from and mimic their 
children — a phenomenon Margaret Mead char- 
acterizes as a pattern of a pffjipmTfif culture.** 
She states that there are three dominant patterns 




71 



of transmitting identity and commitment: l) post - 
figurative where children learn primarily from their 
forebears; 2) cofigurative where both children and 
adults learn from their peers; and 3) prefigurative 
where adults also learn front their children. 

The postal strike ojght to have told somebody 
something! 

The greater consciousness of middle-aged and 
young people Is evidenced by what Dadd Ricsmar 
characterizes as “ ... an atmosphere of . . . ex- 
travagant self-criticism (which) ... has succeeded 
an earlier tendency toward glib self-satisfaction. ” ,n 
I should think a major measure of the cultural im- 
pact of higher education is the extent of self-criti- 
cism and self-consciousness of its products. Of 
course, problems and adversities may become so 
pervasive and intractable that the conscious), css is 
numbed or the problems filtered out in self-de- 
fense. Of course, these observations have less va- 
lidity for students and young people who arc seek- 
ing social mobility and economic security. 

Higher education in the years to come will be 
in great competition with mass media in molding 
the culture of the country. With higher education 
approaching meaningfully open access, if not uni- 
versal attendance, there arc good reasons for pro- 
jecting it as playing the predominant role in culture 
molding. This may not be a gain, at least not 
from the educators' pcispcclive. if Madison Avenue 
and Wall Street take over the educational ‘'enter- 
prise.” 

Conclusion 

Higher education has had a tremendously posi- 
tive impact upon our economy, most demonstrably 
the agricultural economy, htople who have at- 
tended or graduated from college generally earn 
more than those who have not. which permits the 
reasonable inference that higher education increases 
the economic productivity of its student output. 
Generally, it should have been surmised, the more 
years of higher education one bas completed, the 
greater his lifetime earnings and income. (Some 
crafts p*otectcd by trade unions would make one 
wonder about this.) Of course, with the present 
eco-talk about the quality of life, one might further 
surmise that the highly educated are more con- 
scious of cultural and life-quality possibility* in 
society, and thus may he comparatively unhappy 
and sice versa for the less educated (?). 

Higher education has y t for mod reasonably 
well its function of discovering talent, training or 
instructing vvorkerv. and expanding knowledge 



through research — alt to the benefit of the econo- 
my. Research and community (or extension) serv- 
ices have been spectacularly successful in agricul- 
ture. It should be added that research In (he medi- 
cal and biological sciences has been equally suc- 
cessful in advancing nealth science and even health 
technology, narrowly defined, but an increasingly 
disgraceful failure in delivering economical health 
service* to society in general. The deficiency here 
is, in large measure, the fault of (he organized 
medical profession. Its reactionary politics has ob- 
structed public health care and insurance programs. 
Its self-serving guild entropism has discouraged, 
until very recently, increasing the number of trained 
health specialists. The political practices of any 
given profession or pressure group complicate any 
evaluation of higher education's outputs or Impact 
upon the profession or group and those publics 
which the profession or group serves. 

A gross way o) measuring the effectiveness of 
higher education may be to determine how well 
off a particular aspect of the public is which is 
principally served by the products oj higher edu- 
cation, be they workers; research for social, eco- 
nomic, or technological development; or communi- 
ty or extension services. Of course, this obvious 
and somewhat simple method of evaluation can 
create a vicious cycle by Imposing and reinforcing 
unreasonable and unreasonably high expectations 
upon higher education Hv^r. so, in sg ; te of Viet- 
nam, the developing medical scrvtc.s ensis, eco- 
logical apprehensions, deep pockets of poverty, 
and scarcely abated racism, one could say, as Dr. 
Clark Kerr has said. "Higher education in the 
United States is at the pinnacle of its effective- 
ness. , •• , 

However, because of the "collision course’’ dis- 
cussed briefly In the early part of this paper Dr. 
Kerr follows the above quoted statement with, "It 
is also more beset with more fundamental prob- 
lems than ever before in its history.” But even 
these fundamental problems, which I do not think 
create as much of a vrisis as many educators con- 
tend. are more the product of unreasonably high 
expectations created by higher education’s past 
successes than the many deficiencies which are in 
higher education itself. In truth, higher educa- 
tions biggest ptoWem is the adult backlash to post- 
modern youth protest*, demonstrations, and con- 
frontations. The latter ar.- the product of student 
discontents which in substantial measure flow from 
the effectiveness and the high quality performance 
of higher education in its political and cultural im- 
fact upon students. Putting the problem of pro- 
viding better heibh services and greater equality 
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of educational opportunity aside, higher education’s 
biggest problem has been its great success in ele- 
vating student consciousness and sensibility and 
providing the science and knowledge explosion for 
the military-industrial-technological juggernaut. 



1. For example, in 1958 federal support for higher 
education institutions was $730 million. However, ten 
years later it had increased over six-fold up to $4,700 
million In 1968. 

2. David Riesman, 'The Collision Course of Higher 
Education/ 1 The Journal oj College Student Personnel 
363 (November 1969) 

3. Howard R. Bowen, The f inance of Higher Educa- 
tion , 5, (Carnegie Commission, 1968). Dr. Bowen defines 
economic cost as follows : 

The economic cost, on the other hand, reflects 
the true cost to the student (or to society) of his at- 
tending college. It includes the expenses incident to 
education such as books, supplies, transportation, etc. 

It also includes anv expense for board and room, 
clothing, and other living costs over and above what 
would nave been required hnd he not attended col- 
lege. Finally, and most important, it Includes the 
loss of earnings to the student because he has chosen 
not to be in the labor force. 

4. id. at 7. 

5. Change in Higher Education 2 (M arch-April 1970). 

6. Trow, in Daedalus: The Embattled University V9 
(Winter 1970): I. 

7. M. tt 24. 

8. A Chance to Learn: An Action Agenda for Equal 
Opportunity in H icier Educction, A Special Report and 
Recommendations by The Carnegie Commission on High- 
er Education (March 1970). 

9. U. at II. 

10. Clark Kerr, The Pluralistic University in the 
Pluralistic Society , in THE GREAT IOEAS TODAY 
0969); Theodore W. Schulte. Resources for Higher Edu- 
cation: An Economist's IVew, reprinted from The Journal 
of Political Economy 76 (May /June I9'i8) Carnegie 
Commission on Hither Education, 1969. 

If. Schulte, at 32?/. 

12. Jencks d Rresmaft. The Academic Revolution 
( 1968). 

13. Toilet!. "The Governance of t University In a 
Democratic Society: Random Reflections Upon Patlki* 
patory Politks. Black Power, and Law and Order." in 
fmtftgartf! Proceedings, MVigkf State University 33 (Day- 
ton. Ohio 1969). 

14. Income statistics suggest something of the per- 
sonal economk value of a college degree today, fn 1968. 
the median annua) income of head? of households who 
completed four or more years of college was $12 288; 
for tWe with one to three >tars of college, it was $9,483: 
and for high school graduates. $8,583." Cwrrnr Reading. 
The PuNk Interest No. 19. 133 (Spring 1970). 

15. Caryl P. Haskins. "The Humanities and the Na- 
tural Sciences." American Scientist 23 (fanu^ry February 
1970) :58 

16 Id. at 28 



17. Ibid. 

18. “The Lonely Frontier of Reason/ 1 The Nation, 
May 27, 1968. 

19. Kerr, at 23. 

20. Conway, “Styles of Academic Culture,” in Daeda- 
lus: The Embattled University 99 (Winter 1970) :43. 

21. Id. at 44. 

22. George Kaleb. “The Campus and Its Critics/’ 
Commentary 47 (April 1969) : 40-41. 

23. The New York Review of Books ?3, October 
9, 1969. 

24. Ibid. 

23. Id,, at 22 (emphasis added). 

26. Schulu, at 337, writes: “We need to remind our- 
selves that there are still some children who are not com- 
pleting the elementary grades; and what 1 j more imror 
tant is the underinvestment in the quality of elementary 
schooling, especially in many rural areas . . . <T)he pri- 
vate rates of return to white males after personal laves 
in 1958, are 28 per cent tor high school graduates and 
14.8 per cent for college graduates.” 

27. Theodore Rosrak, The Making of a Counter Cul- 
ture (1970). 

28. Mukhins. The Democratic Dihmma, The Gotles- 
man Lectures Uppsala University. (Stockholm. Sweden. 
1951). 

29. Friedenberg. “Legitimate Violence/* The Nation 
206. Dune 24, 1968): 822. 

SO. Glarer, “Student Polices in a Democratic So- 
ciety” The American Scholar 36. (Spring. 1967). 

31. “The Role of the University: Ivory To act. Serv- 
ice Station, or Frontier Post?** Daedalus: The Embattled 
University 99 (Winter 1970) : 75. 

32. /<#. at 83. 

Of course it is important to heed the warning of John 
H. Schaar and Sheldon S. Woliti that higher education 
must avoid the “deeper retrea. into the sanctuary of in- 
geniously obscure research”-— while mumbling incanta- 
tions about higher value). “ilducation and the Tech- 
nological Society,” The New Fork Review of Rooks 13 
(October 9, 1969) :3. 

3). Trow, supra note 6, at 2. 

34. The Chronicle of Higher Education, March 9. 
1970. on page 3 repor':! discussion of financing higher 
education. It fndkated that in the fast President Nixon 
applauded a full-cost tuition, lone-term loan, open market 
approach to higher education. Howard R. Bowen. Presi- 
dent of Claremont University Center, stated that such an 
approach would make colleges and universities subservb 
enl to every whim of the customers. "Moreover.” he 
said, “at a time when at are trying to Wing millions of 
minority and poor people into the maimtream of Ameri 
can sxkiy. it would seem perverse to raise the pore 
of higher education and to offer it only to those will- 
ing lo go heavily into debt " 

Mr. Bowen advocated instead relatively low- tuitions 
" . . . veering toward the free -puNk -education model 
rather than the full-coM model ” 

35. "flow Shall We Finance Higher Education?” The 
Public Interest t Spring 1970) :98. 

36. Id. at 100. 

37. Ibid 

38. Supra, note 13. at 34 

39. Stead, “Youth Revolt: The Future rs Now," Sat- 
urday Rniew 53. (Jammy 10. 1970). 

40. Riesman. “ The Lonely Crowd* 20 Y*ears After," 
Encounter 33, (October 1969). 

41. Ken. New Challenges to the College and UmVrr* 
rity (Carnegie Commission on Higher Education. 1969). 




SOME APPROACHES TO MEASUREMENT 





Alexander W. Aetln 

Is Director of Research for the 
American Council on Educe* 
lion In Washington, D. C. Prior 
to this position, he was Dlrec* 
tor of Research of the National 
Merit Scholarship Corporation 
In Evanston. Illinois. 8!nce 
1960 he has been conducting 
research on the comparative 
effects of different types of 
college environments on e stu* 
dent's development. 

In 196S he received the Amerl* 
can Personnel and Guidance 
Association's Award for Out* 
standing Research for his 
studies of college characters* 
tics and college effects. 

he Is directing the 
ve Institutional Re* 
ogram of the Amerl* 
can Council on Education, an 
on-going longitudinal study In* 
volvlng a national sample of 
some 400 colleges and uni- 
versities. Dr. Astln was 
awarded his Ph.D. at the Uni- 
versity of Maryland. 



Currently. 
Cooperatl 
search Pr 



"Sfudenf output* vo uld Include measure* ot the student’s 
achievement*, knowledge, sJ kill*, value*, altitude*, etplta • 
f/ont, Interett*, dally ectlvllle*, and contrlbullon* to eoclety." 
" Student Input* are the talent*, tklll*, atpltatlont, and other 
potential* tor growth and learning that a new ttudent bring* 
with him to college. Tnete Input* ere, In a sente, the raw ma- 
terial* with which the Institution ha* to work/' 

"The college environment refer * to any aspect of the higher 
educational Inttltutlon that I* capable ot affecting student 
output*. Broadly speaking, (he term Include* variable* such 
as administrative policies and practices, peer association*, 
and other attribute* ot the college enperlence that might 
effect the student'* development." 

"No management Information system Is of much use unless 
the causal connection* between environmental variables and 
output variable* are known." 





Measuring Student Outputs in Higher Education 

ALEXANDER W. ASTIN 



75 



Any consideration of how best to measure the 
outputs of higher education necessarily requires 
that one clarify just what is meant by "outputs." 
For the purposes of this paper, 1 shall use a con- 
ceptualization of student outputs that has char- 
acterized recent longitudinal research on the im- 
pact of colleges on their students (Astin. 1970). 
While this view may be foreign to some readers, 
particularly lo those whose backgrounds are in 
economics or related fields. I believe that U is 
useful in elucidating certain basic issues which 
might otherwise be overlooked. 

The System of HlQher Education 

Briefly, in this view, the process of higher edu- 
cation is regarded as comprising three conceptually 
distinct components: student outputs, student in- 
puts, and the college environment, Student outputs 
refer to those aspects of the student's development 
that the college cither influences or attempts to 
influence. Although these outputs can be expressed 
at very high levels of abstraction (for example. 
" . . . the ultimate happiness and well-being of the 
individual"), we shall limit our consideration of 
the problem to those relatively immediate outputs 
that can be operationalized. Specifically, then, stu- 
dent outputs would include measures of the stu- 
dent's achievements, knowledge, skills, values, at- 
titudes. aspirations, interests, daily activities, and 
contributions to society. Other terms that arc sonic- 
times used to refer to student outputs arc depend- 
ent variables, criterion variables, ouiconrc variables, 
and educational objectives. 

Student inputs are the talents, skills, aspirations, 
and other potentials for growth and learning that 
the new student brings with him to college. These 
inputs arc. in a sense, the raw materials with which 
the institution has to work. Some inputs may be 
simply "pretests** on certain student outputs (scores 
on college admissions tests, for example), while 
others (sex and race, for example) may be rela- 
tively static personal attributes. Inputs can affect 
the outputs either directly or through interaction 
with environmental variables (below). U should 
be pointed out that economists and others inter- 
ested in systems analysis or management informa- 
tion systems typically use the term "input" in a 
much broader sense than the one proposed here. 
Usually, this broader use includes what I have 



termed environmental variables (below) as well 
as student inputs. In assessing the outputs of higher 
educational institutions, however, it is useful, if not 
essential, to differentiate conceptually and opera- 
tionally between measures of student inputs and 
measures of the college cmironment. 

The college environment refers to any aspect 
of the higher educational institution that is cap- 
able of affecting student outputs. Broadly speak- 
ing, the term includes variables such as administra- 
tive policies and practices, curriculum, faculty, 
physical plant and facilities, teaching practices, peer 
associations, and Other attributes of the college ex- 
perience that might affect the student's develop- 
ment. These environmental variables can, pic- 
sumably. be changed or manipulated through re- 
allocation of resources. 

The relationships among the three components 
of the model are shown schematically in Figure I. 
Note that student outputs can be affected by both 



Figure I. 

The Three Components of the Model 




environmental variables (Arrow B) and student 
input variables (Arrow C). Moreover, as Arrow* 
A indicates, college environments can be affected 
by the kind* of students who enroll. In addition 
to these "main'* effects of environments atrd inputs 
on outputs, there may be interaction effects involv- 
ing student inputs and co'!ege environments. As 
the diagram suggests, there are two types in inter- 
action effects: those in which the effect of input 
on output is different in different college environ- 
ments (AC), and those in which the effect of the 
college ensironmem is different for different types 
of students (AB). 
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The maltcr of assessing ihc outputs of higher 
education involves two basic problems. The first 
is that of defining and measuring the relevant out- 
put variables; this will be the main subject of this 
paper. The second, closely related, task is that of 
determining the effects of environmental and stu- 
dent output variables. This latter problem, which 
Involves primarily experimental design and statis- 
tical mcthodolgy, is dealt with at length in an- 
other paper (Astin, 1970). Suffice it to say that, 
no maltcr how- elegantly or appropriately the out- 
put variables are measured, no management infor- 
mation system is of much use unless the causa! 
connections between environmental variables and 
output variables arc known. In order to make 
trustworthy judgments about these causal relation- 
ships. it is necessary first to conduct longitudinal 
studies that incorporate data on student inputs, stu- 
dent outputs, and college environmental character- 
istics (Astin & Panos. 1970). 

Conceptual Outputs and Output Measure* 

The measurement of any educational output 
ordinarily begins with a statement about some as- 
pect of the student's development that is considered 
desirable or important. This verbal statement, or 
conceptual output, originates in the value system 
of the person making the statement. The task in 
developing an appropriate output measure is to 
operationalize this conceptual output In some way 
( vstin. 19**). The process of operationalizing 
typically involves collecting empirical data which 
arc judged to b*' relevant to the conceptual output, 
tn some instances, the raw data already takes the 
form of a “measure” (dropping out versus slaying 
in college, for example), whereas in others, the 
raw data must be combined or manipulated in some 
way in order to derive the final measure (answers 
to multipie-choice questions on an achievement 
lest, for example). 

In attempting to “evaluate" an output measure, 
or set of measures, one must keep in mind the 
distinction between conceptual outputs and output 
measures. The appropriateness of ary empirical 
measure can be judged onty in 'erms of its ap- 
parent relevance to the conceptual output. In con- 
trast. the conceptual criterion itself — being de- 
rived basically from value judgments — is not 
subject to lire same sort of analysis. 

Although we cannot differentiate operationally 
between the conceptual output and the output 
measure, the construct of fonor/>»»<»f output is im- 
portant for at least two reasons. First, the relevance 



of any output measure and the rationale for its be- 
ing selected arc belter understood if the conceptual 
stage is documented. Second, because the con- 
ceptual output often implies something more than 
the actual output measure, it may prove to be a 
valuable source cither of future output measures 
or of improvements in existing ones. 

8tngl* or Multiple Meaiur**? 

Any discussion of the measurement of educa- 
tional outputs must give some attention to the 
question of using multiple measures versus a sin- 
gle, “overall" measure of educational progress. Al- 
though a single output measure possesses certain 
obvious advantages because of its conceptual sim- 
plicity and computational convenience, it is un- 
realistic as well as misleading to reduce college 
impact to a single output measure. Therefore, in 
discussing the various problems associated with 
measuring student outputs. I have assumed that 
there are many outcomes, both planned and un- 
planned, which must be measured if an adequate 
and useful assessment Of institutional impact is to 
be carried out. This assumption is based on a 
belief that even within a relatively homogeneous 
group of undergraduates, individuals can vary 
greatly in Ihc importance they attach to any given 
educational outcome. Accordingly, this paper will 
not treat the problem of assigning relative weights 
to various alternative outcomes. Subjective weight- 
ing in terms of degree of "importance” requires 
value judgments that are more appropriately the 
province of the students, educators, planners, legis- 
lators. and others directly concerned, tn other 
words. I have assumed that the conceptual and 
methodological problems associated with identify- 
ing and measuring relevant educational outcomes 
can be isolated from the more personal and value- 
laden problem of deciding what degree of impor- 
tance should be attached to any particular out- 
come. The challenge to the researcher, then. Is to 
make sure that his coverage of student outputs is 
broad enough to meet most of the major concerns 
of the various constituent groups. 

Although it is important that any battery of 
output measures reflect the interests of as many 
persons as possible, an adequate assessment of the 
outcomes of the college experience cannot be 
limited solely to outputs that are either desired 
or intended. The unintended outputs of “side ef- 
fects” of the college experience must be consid- 
ered. loo. 

When we talk about side effects in medicine, 
they are usually regarded as undesirable; but the 
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side effects of attending college may or may not 
be desirable, depending on the particular side ef- 
feci and the value system of the individual imme- 
diately concerned. A new college curriculum, for 
example, may have its intended effects on the stu- 
dent's cognitive abilities, but it may also have un- 
planncd-for effects on. say. the student's vocation- 
al and career plans, tn short, as long as the pos- 
sibility exists that a particular output measure is 
affected by the student's college experiences, that 
measure should not be rejected as “irrelevant ' sim- 
ply because it docs not fit in with sonic a priori 
educational plan. 

A Simple Taxonomy of Output Measure* 

Because the number of possible output meas- 
ures is very large, some sort of ro.umomy of such 
measures is essential. Taxonomies are useful not 
only for classifying existing output measures, but 
also for suggesting additional measures. As a be- 
ginning, I would propose that output measures be 
classified on the basis of three attributes: the type 
of outcome the type of data, and the time span 
involved. 

Typa ol Outcome 

Behavioral scientists usually classify measures 
of human performance into two broad domains: 
cognitive (sometimes called 'intellective") and 
noncognitive (sometimes called "affective"). Cog- 
nitive measures have to do with behavior that re- 
quires the use of high-order mental processes such 
as reasoning and logic. Of all the possible meas- 
ures of performance that one might devise for 
evaluating student outputs, those involving cogni- 
tive learning and the development of cognitive skills 
are probably the most relevant to the educational 
objectives not only of students, but also of faculty, 
administrators, trustees, parents, and others con- 
cerned with higher education. 

Noncognitive. or affective, measures have to 
do with the student's attitudes, values, self-concept, 
aspirations, and social and interpersonal relation- 
ships. Although the number of possible nonoogni- 
tive outputs is very large, techniques for measuring 
such outputs are not as far advanced as arc those 
for measuring cognitive outputs. 

Type ot Data 

The second dimension of taxonomy refers 
to the operations which are performed in order to 
obtain nKasurcments of (be cognitive or affective 
outputs under consideration. Again, two broad 



classes can be identified: psychological data relat- 
ing to the interna! states or "traits'’ of the indi- 
vidual. and behavioral data relating to the observ- 
able activities of the individual. The measurement 
of psychological phenomena is usually indirect, in 
the sense that the investigator infers sonic under- 
lying state within the individual from his responses 
to a standard set of test questions. Behavioral 
measures reflecting transactions between the per- 
son and his environment are usually of intrinsic 
Interest. Since behavioral (as opposed to psycho- 
logical) measures typically involve observing the 
individual In his environment, such measures might 
also be termed "sociological.*’ 

Any student output measure can be classified 
simultaneously by the type of outcome involved 
and by the type of data uscu as shown in Table I. 
Each cell provides examples of different types of 
outcome measures obtained using different types 
of data. The cell in the upper left of Figure I, for 
example, includes cognitive outcomes that are often 
measured in terms of the student's grade-point av- 
erage or his performance on multiple-choice tests 
of ability and achievement. The undergraduate stu- 
dent's knowledge, basic skills, and critical think- 
ing ability are often assessed by means c’ batteries 
such as the Graduate Record Examination (GRE). 
Achievement in specialized fields can be measured 
by means of the Advanced Tests on the GRE. 
and general achievement and intelligence by means 
of the GRE Area and Aptitude tests. One aspect 
of cognitive ability for which psychological meas- 
ures are probably still not adequate is creative 
ability. Gily a few exist (Taylor Barron. 1963). 

The upper right cell of Table I includes psy- 
chological measures of noncognitive or affective 
itates: the student's ambition, motivation, self-con- 
cept, as well as Hs subjective feelings of satisfac- 
tion and well-being. Most of the research on col- 
lege impact that has been conducted so far has 
been concerned with psychological measures of 
affective slates. Thus, of the more than 1.000 
studies of college impact recently reviewed by 
Feldman A Newcomb (1969). only a handful used 
measures of cognitive outcomes and. of these, vir- 
tually all used psychological rather than behavioral 
measures. This bias is. to some extent, probably a 
matter of logistical convenience. Psychological 
measures of affective slates — attitude*, values, 
career plans, and the like — are simple and inex- 
pensive to administer to undergraduates before, 
during, and after they attend college, whereas meas- 
ures of cognitive outputs and measures of be- 
havioral or sociological outputs ordinarily requite 
considerable time and expense to administer. 
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Table 1, A Taxonomy of Student Output 
Measures in Terms of Type of Outcome and Type of Data 



Type of Outcome 



Cognitive Aff«ctlvv 



Knowledge 
General Intelligence 
Critical Thinking Ability 
Basic Skills 
Special Aptitudes 
Academic Achievement 


Sell-concept 

Interests 

Values 

Attitudes 

Beliefs 

Drive for Achievement 
Satisfaction with College 


Level ol Educational 
Attainment 

Vocational Achievements: 
level of responsibility 
Income 

awards ol special 
recognition 


Choice of * Major or Career 
Avocations 
Mental Health 
Citizenship 

Interpersonal Relations 



The lover left cell in Table 1 gives examples 
of behavioral or sociological measures of cognitive 
outputs, Basically, this category refers to outputs 
which reflect the behavior of the student (or for- 
mer student) in society and which ordinarily re- 
quire cognitive skills. Presumably, these ical-lifc 
achievements represent the behavioril manifesta- 
tions of the cognitive traits listed in the cell above 

The fourth cell, located in the lower right 
quadrant ol Table I. includes sociological or be- 
havioral features of the individual's development 
that reflect primarily affective states. Under “avo- 
cations." for example, one might include the 
anKHint of time spent by the person in various 
recreational pursuits. ‘‘Citizenship * might be mani- 
fested in the amount and quality of participation 
in community activities. the earning of special 
awards for service to the community, cv, on the 
negaiive side. welfare or arrest records. 

It should be pointed out that the two dimen- 
sions comprising Table 1 are really more coniinua 
l v .an true dichotomies. For example, a person's 
earned income and f>b status (which would be 
subsumed under “vocational achievements'* in the 
cognitr* domain) may involve his m>n<x>gniiive 
or personality traits. 

With respect to the two tvpes of data f psycho- 
kgkal and behavioral) defining the two rows of 



Table I. it is important to beep in mind that psy- 
chological measures are less likely to be r«?gardcd 
as socially relevant outputs than are the behavioral 
measures. Nevertheless, some psychological meas- 
ures “acquire" behavioral significance as the result 
of empirical research. Measures of general intelli- 
gence and academic ability, 6 ; example, have ac* 
quired behav ioral significance because longitudinal 
research has shown that persons with high scores 
on such measures arc usually successful vocation* 
ally and socially Herman ft Oden. 1947; Nichols 
A Astin. 1966). The connection between psycho- 
logical and behavioral measures in the affective 
domain, however, is much less well-established. 
Perhaps the only instance in which psvchological 
measures of a f feclive outputs are known to bear 
more than ^ trivial relationship to subsequent be- 
havioral pbcnon>cna is interest measurement. Meas- 
ured vocational interests arc closely associated with 
later career development and success over many 
years (Strong. 1955). 

Ttm# Span Co rfrH bf tho Mmuta 

The four cells shown in TaNe I could be ex- 
tended into a third dimension to portray the tem- 
poral aspects of the output measurement being 
used. Although it is not often considered in dis- 
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cussions of educational outputs, time is a funda- 
mental consideration. Is it more appropiiate to 
measure the immediate outputs of the college ex- 
perience — that is, those that take place after only 
a brief span of time — or the outputs which show 
the long-term effects of the college experience? 

Most educational institutions are designed with 
the hope of producing long-term rather than short- 
term changes. The goals stated in college catalogs, 
for example, imply that the institution is priitu/dy 
concerned with making an impact on the student 
that will last throughout his lifetime. The college, 
it would seem, tries to provide experiences which 
will assist the student in making the fullest pos- 
sible use of his talents and in becoming an effec- 
tive, responsible member of society. Presumably, 
such effects will, in turn, result in a more satisfy- 
ing and rewarding life. 

For many prospective college students, how- 
ever, such long-term effects are too remote and 
too difficult to comprehend These students are 
primarily interested in much more immediate goals 
— their actual experiences during the undergradu- 
ate college years — rather than in how these ex- 
periences will affect their later development. Edu- 
cators frequently overlook the fact that the two, 
four, or eight years of college represent a sizable 
proporlion of the student's total life span. To him, 
then, college experiences are important in and of 
themselves, not merely in what they will mean later 
on after the student leaves college. 

ft seems likely that psychological measures, 
particularly of cognitive outputs, are most relevant 
for the college years, whereas behavioral measures 
are more suitable for assessing both short- and 
long-term effects. The point to keep in mind is 
that the appropriateness of any output measure — 
whether it be cognitive or affective, whether it be 
derived from psychological or from behavioral 
data — is determined in part by the period of time 
spanned by the measure. Clearly, there are valid 
arguments for using both short-term and long-term 
measures. 

The importance of timing in the development 
of output measures can be illustrated with a hypo- 
thetic .1 example. Let us assume that a student from 
a small rural high school enrolls in a highly selec- 
tive and academically competitive institution. His 
initial experiences result in a variety of relatively 
short-term or immediate outputs, both psychologi- 
cal and behavioral, both cognitive and affective. 
Affectively, the student feels anxious about possi- 
ble academic failure, hostile and competitive 
toward fellow students, and proud of the institution. 
On the behavioral side, he devotes more time to 



study and less to social activities, and he becomes 
more intellectually aggressive toward his fellow 
students and his instructois, more inclined to argue 
with them. Such affective changes might be as- 
sessed by means of inventories or questionnaires 
administered at the time of matriculation and again 
after a few months in college. On the cognitive side, 
our hypothetical student becomes much more 
knowledgeable about certain subjects as a result 
of his increased devotion to study. Such knowledge 
would, presumably, be reflected in followup 
achievement testing. Behavioral manifestations of 
cognitive changes take the form of his being elected 
to honorary societie;- and receiving awards for spe- 
cial achievements (essays, authorship of publica- 
tions, etc.). 

The longcr-range effects of attending the high- 
ly competitive institution can also be classified by 
type of outcome and by type of data used. To re- 
turn to our hypothetical student, we find that in 
the affective domain his four years in the highly 
competitive institution have fairly long-lasting ef- 
fects on his self-esteem, as well as on his feelings 
of competitiveness, anxiety, and inferiority. These 
psychological changes are also manifested behavi- 
orally in the amount of time he devotes to his job 
and by the extent t^ which he competes with 
others on the job. Cognitive outcomes of attend- 
ing the highly selective college are reflected in his 
successfully completing graduate school and, later, 
in his achieving a high-level position, a high salary, 
and special awards from his professional society for 
outstanding achievement. 

ft should be pointed out here thai psychologi- 
cal measures of cognitive outputs, such as perform- 
ance on standardized tests, are usually not ob- 
tained once a person leaves formal higher educa- 
tion. In fact, the person who holds a college de- 
gree normally does not have to take cognitive tests 
that arc otherwise required by the military, indus- 
try, and the civil service, But there is no leason 
wTiy cognitive testing could not be used at any 
time following college, given adequate resources 
and the subject's willingness to participate. 

Relative Versus Absolute Measures 

Our discussion of Figure 1 has already sug- 
gested that the problem of defining educational out- 
comes is inextricably bound up with the method- 
ology employed to secure the actual measure. 
Methodology involves not only the operations that 
are performed to gather the raw data but also the 
manipulations cf the raw data by which the final 
“measure” is derived. In the field of achievement 
testing, for example, (he common practice is to 
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devise a list of multiple-choice test items; adminis- 
ter them to a sample of students; count the number 
of items answered correctly by each student; and 
then calculate a derived measure which is expressed 
in terms of a standard score, a percentile score, or 
some other relative value. That is, a person’s 
final score reflects only how he has performed rela- 
tive to others. 

In achievement and aptitude testing, relative 
measures are used almost universally as measures 
of educational outcomes. But they present some 
potentially serious problems. Such measures indi- 
cate nothing about the student’s absolute level of 
performance. They give no direct information as 
to how many items the student answered cor- 
rectly, how difficult the items were, or what the 
student’s test performance implies about his po- 
tential for performing well on the job, profiting 
from further education, and the like. More impor- 
tant, such relative measures offer no way of re- 
flecting changes in the student’s performance over 
a period of time. Thus, it is possible for a stu- 
dent’s absolute (i.c., actual) peiformance or com- 
petence to improve considerably over a period of 
time, while his relative performance remains the 
same, or even declines during the same period. 

There are several explanations as to why spe- 
cialists in educational measurement have a predi- 
lection for rela ive rather than absolute measures 
of performance. One argument for derived scores 
(particularly standard scores) is that they possess 
certain statistical properties that make them more 
suitable for some types of analyses. A more subtle 
explanation involves the meritocratic bias of the 
culture and, in particular, of the educational sys- 
tem. Derived scores, which are an indication of 
the person’s relative ordering among his peers, 
provide a convenient means of identifying the 
most talented persons for purposes of recruitment 
into jobs, graduate and professional schools, and 
other areas where competition for a finite pool of 
talent is strong. Nevertheless, for purposes of as- 
sessing the impact of any college on the student's 
development, derived or relative measures of stu- 
dent outputs appear to have very limited useful- 
ness. 

There arc several possible approaches to the 
problem of developing measures of absolute per- 
formance. If the measure is based on the types 
of multiple-choice items usually found in aptitude 
or achievement tests, perhaps the most straight- 
forward approach is simply to record the number 
of items correctly answered. Change or growth in 
the student’s development can thus be measured 
in terms of increases in the number or percentage 
of such items correctly answered. One useful 



elaboration of this approach is to develop ex- 
pectancy tables that show the probability of vari- 
ous events (performing well on the job, for ex- 
ample) as a function of (he number of items cor- 
rectly answered. Change or growth can then be 
measured in terms of increases in these probabili- 
ties over time. 

Another approach to the development of abso- 
lute measures of performance is to label particular 
points on the distribution of scores (whether they 
be raw or derived) in terms of the level of per- 
formance typical of that point. For example, if 
one were interested in using an output measure 
of general intellectual achievement, the lowest 
scores would indicate borderline literacy, and the 
highest scores would correspond to the level of 
intellectual achievement normally required of stu- 
dents pursuing Ph.D.-level graduate education. The 
significance of particular scale points would be 
made even clearer, if examples of actual items were 
used to show the most difficult types of items 
passed by the majority o f people scoring at a par- 
ticular point on the scaw. 

Absolute measures of performance probably 
provide the ideal basis for making educational ob- 
jectives explicit. Let us assume, for example, that 
we have constructed a measure of cognitive per- 
formance that covers a wide range. This measure 
is represented by the distribution shown in Figure 
2-a (we have made the distribution “normal” in 
shape, but there is no necessary reason why the 
actual distribution of raw scores in the population 
could not assume some other shape). Two major 
cutting points on the score distribution have been 
identified: “borderline literacy,” at the low end of 
the continuum, and “Ph.D.-level performance,” at 
the high end. Let us assume that the normal dis- 
tribution shown in Figure 2-a represents the scores 
of the total population of potential college-going 
students at the point of graduation from high 
school. Note that only a very small fraction of the 
population is performing at the Ph.D. -level at the 
time of entrance to college, but that a substantial 
proportion is performing at or below borderline 
literacy (the cross-hatched areas of the distribution 
above and below these two points are arbitrary; 
they have been drawn as shown simply for illus- 
trative purposes). The desired educational out- 
put — the goals ot the higher educational system, 
if you will — can be specified in terms of changes 
in the characteristics of the distribution. 

Although an almost infinite number of such 
changes might be desired. Figures 2-b, 2-c, and 
2-d are examples of only three basic types of 
changes. The solid lines in each of these three fig- 
ures show the desired shape of the distribution 
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after four years of college (the student output); 
the dotted line which is superimposed on each fig- 
ure shows the distribution at the point of matricula- 
tion (the student input). The first of these hypo- 
thetical changes in the performance distribution 
(Figure 2-b) involves an upward shift in the stu- 
dents* mean performance, only. Note that the en- 
tire distribution has simply shifted to the right and 
that the shape or dispersion of people remains un- 
changed. One might refer to this as a sort of 
"democratic** or "egalitarian 1 * plan. 

An alternative educational outcome is por- 
trayed in Figure 2-c. Here the proportion of the 
students performing at or near the Ph.D.-level has 
been substantially increased, while the scores of 
those at the lowest parts of the distribution remain 
almost unchanged. Jhi% type of change, which is 
concerned primarily with increasing the proportion 
of very high-performing students, might be char- 
acterized as an "elitist” plan, in* the sense that em- 
phasis is placed on increasing the number of very 
high performers; and there is relatively little con- 
cern with improving performance at the lowest end 
of the continuum. This type of cognitive objective 
has characterized much of American higher educa- 
tion in the past. 



A third alternative outcome, shown in Figure 
2-d, is concerned primarily with minimizing the 
proportion of low performers. Here the number of 
persons performing at or near borderline literacy 
is greatly reduced, but the number of performers 
at the high end of the distribution changes only 
slightly. Since it is concerned primarily with the 
eradication of illiteracy, this approach might be 
labeled as “remedial.’* 

Many other changes in the distribution of 
scores are, of course, possible. My intention here is 
to emphasize that the specification of any educa- 
tional output requires more than just the develop- 
ment of a measure such as "cognitive performance"; 
it requires as well the specification of what changes 
are desired in the distribution of the target popula- 
tion. Note that the desired change as specified in 
figure 2-b involves an alteration only in the stu- 
dents* average performance, whereas the changes 
shewn in the last two distributions involve altera- 
tions in the variation of scores as well. The "elitist" 
approach results in an increase in score variation 
by stretching out the scores at the high end of the 
distribution; the “remedial" approach, on the other 
hand, results in a decrease in the dispersion of the 
scores because of major increments to the scores 
of the lowest-perforrri:ig individuals. 



Figure 2: Hypothetical Distribution of Intellectual Ability 
in the Population of Entering College Students (A), and 
Three Possible Outputs of the Higher Educational System 
Expressed in Terms of Changes in the Shape of the 
Distribution (B, C, & D). 
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Determining whether or not the desired chang- 
es in performance have taken place involves ad- 
ditional problems in measurement. It is not enough 
just to ascertain if the mean performance of all 
individuals has improved, or even if the variation 
(the standard deviation) has increased or decreas- 
ed, (An increase in dispersion or standard devia- 
tion could result from a deterioration in the per- 
formance of the lowest-scoring individuals as well 
as from substantial gains in the scores of highest- 
performing individuals.) One method is to develop 
several dichotomous measures based on critical cut- 
off points in the distribution (“borderline literacy’' 
and “Ph.D.-level performance” represent just two 
such cutoff points). It might also be useful to study 
changes in the shape of the distribution as reflected 
in measures of skewness and kurtosis. 



The input-environment-output model of stu- 
dent development in higher education (Figure 1 ) 
is designed primarily to aid the investigator in as- 
sessing the impact of different types of environ- 
ments on relevant student outputs. Student input 
data are needed in order to measure changes in 
the student’s development and to make statistical 
adjustments for the non-random distribution of 
students among institutions; environmental data are 
needed in order to identify those institutional at- 
tributes which affect the student outputs under in- 
vestigation. Longitudinal studies involving these 
three types of data can yield information on the 
causal connections between environmental and stu- 
dent output variables. Such research information 
can, in turn, provide the basis for developing models 
of the higher educational system which can be used 
in planning. 

Aside from the methodological problems in- 
volved in studyi \g environmental effects on stu- 
dents, several considerations must be taken into 
account in designing or modifying environmental 
variables in order to oroducc changes in the distri- 
bution on some student output measure. First, the 
target population must be defined. D* > the distri- 
bution shown in Figure 2 a, for example, ion r only 
to the college-going population, or does it include 
as well high school graduates who do not go on to 
college? Even if the target population is defined 
to include only those who will actually go to col- 
lege, the investigator must be prepared to deal with 
temporal changes in the characteristics of this pop- 
ulation. The number of available scholarships and 
loans may increase; recruitment activities may be 
intensified; and other changes may occur which 
affect students’ decisions to go to college. In view 



of these possibilities, defining the population to in- 
clude all potential college students* has the ad- 
vantage of permitting the investigator to include 
recruitment practices and admissions policies 
among his array of environmental variables and 
thereby to estimate how various changes in these 
policies and practices would affect the distribu- 
tion of the total population on the output measure. 

Second, one must be aware that certain changes 
in a person’s output performance may be develop- 
mental or maturatioml in nature. That is, no mat- 
ter what sort of college experience a person has, 
and even if he never attends college, his cognitive 
abilities and other output performances will under- 
go change with time. Thus, if Figure 2-a shows 
the output distribution in the population at one 
poirt in time, the distribution will probably be dif- 
ferent at some subsequent point in time, even if 
none of the population attends college. For exam- 
ple, it seems likely that many persons at the low 
end of the distribution will show a substantial de- 
cline in cognitive performance after high school if 
they receive no further formal education. Clearly, 
one needs estimates of these expected changes in 
output among non-college students in order to as- 
sess the potential impact of changes in recruitment 
or admissions policies. 

Perhaps the biggest problem in using student 
output measures for educational modeling is posed 
by the existence of multiple output measures. The 
statistical problems in dealing simultaneously with 
several output measures greatly complicate the in- 
vestigator’s analytic task.** In addition, because 
persons differ so greatly in the importance that they 
attach to different outputs, what may be seen as an 
educational “side effect” by om person may icp- 
resent the principal education;! objective of an- 
other. Under these circumstances, the investigator's 
model must be able to accommodate several out- 
put measures simultaneously, so that any decision 
to manipulate environmental variables in order to 
produce changes in certain output variables can be 
made with a knowledge of how other output var- 
iables are likely to be affected. 



* The population of "potential 0 students would in- 
clude all those who could be persuaded to attend college 
under conditions of maximum incentive. Defining what 
constitutes “maximum incentive,” given finite resources, 
is, of course, an important unsolved problem that merits 
a major research effort. 

** Because they weight dependent (output) variables 
basically in terms of their predictability ratner than their 
actual importance, statistical techniques such as canoni- 
cal correlation and multiple discriminant analysis are 
probably inappropriate for dealing with multiple output 
measures; separate analyses for each output measure arc 
probably more appropriate under these circumstances. 
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In this paper, I have attempted to deal with 
several problems related to developing measures 
of student outputs in higher education. The follow- 
ing are among my major points: 

1. Student outputs can be unde rstood if they are 
viewed in relation to the total higher educational 
system. Specifically, student outputs should be con- 
sidered as part of a three-component model com- 
prising student outputs, student inputs, and char- 
acteristics of the college environment. Educational 
planning requires a knowledge of how outputs are 
affected by environmental variables. Such effects, 
however, cannot be determined without informa- 
tion on student inputs. 

2. Because of great variations in the values and 
objectives of different persons, any attempt to 
develop a single “overall” student output measure 
is unrealistic. Rather, the investigator must seek 
to develop a battery of measures that is sufficient- 
ly broad to satisfy the major concerns of a substan- 
tial number of students, educators, and planners. 
In addition, provision should be made to include 
measures of possible “side effects.” 

3. A preliminary taxonomy of student-output 
measures would include the following three dimen- 
sions: the type of outcome (cognitive versus af- 
fective), the type of data (psychological versus 
sociological), and the temporal aspects of the meas- 
ure (short-term versus long-term). This taxonomic 
scheme should prove useful, both for classifying 
existing measures and for identifying gaps where 
additional measures are needed. 

4. Although relative or derived measures are 
widely used in educational research, particularly 
in the measurement of cognitive outputs, such 
measures present serious conceptual problems 
which limit their value for modeling and planning. 
Whenever possible, the investigator should strive 
to develop absolute rather than relative measures of 
student outputs. 

5. The use of output measures — whether rela- 
tive or absolute — in educational modeling and 
planning requires that the desired population 
changes, in the distribution of scores on each 
measure, be specified. In planning such changes, 
the investigator must consider changes in the shape 
of the performance distribution as well as changes 
in the mean and variance. 
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“ The legislator, university administrator, or educational re- 
former wants to know . . . what will happen If a conscious, 
concerted effort Is made to try to get more of a particular 
output." 

" When my son Christopher got all A’s In first grade but found 
school repressive and dull, the argument that his grades Indi- 
cated a successful and lucrative future was little consolation 
to him or to me." 

"Most of the cost of higher education Is the wages foregone 
while In college." 

"It Is beginning to appear that the most overrated argument 
for public subsidies of higher education Is that made on the 
grounds that there are externalities or spillovers from an Indi- 
vidual student’s education which benefit us all." 

"Many higher education activities are billed as benefiting 
‘the public,’ when, In fact, they merely represent the effective 
lobbying of spechl Interests, Including university far.ulty." 
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Public Service Outputs of Higher Education: 
An Exploratory E^say 

JOHN E BRANDL* 



This paper is devoted to beginning a discussion 
of the public service outputs of higher education. 
These are defined as non-rcsearch benefits (or 
costs) accruing to persons other than students and 
employees of institutions of higher learning. 1 For 
some such activities an institution is reimbursed 
directly, for others, not; so the discussion is not 
limited to the latter cases, despite the fact that 
public service seems in common usage to carry 
connotations of generosity or altruist/ 1 . There is 
virtually no literature on this question — which 
helps to explain both the timid goals and the 
modest achievements of the paper. Anecdotes on 
the civic-mindedness of the college educated.- and 
rhapsodies on the contributions to American agri- 
culture of Land Grant Colleges' 1 nearly exhaust 
what has been said. 

The reason why we ore worrying about the out- 
puts of education in this symposium is that educa- 
tion is a government-dominated field in which gov- 
ernmental involvement has traditionally been justi- 
fied on the ground* that in three important ways 
the market bieaks down. The three are: (1) ex- 
ternalities of education, i.e., societal benefits flow 
from an individuals schooling; (2) the public 
good nature of the product of research, i.e., ideas 
are available to the entire society which therefore 
can be called upon to finance their production; and 
(3) the neutrality of the market on the question 
of egalitarianism, i.e., education is often seen as 
a mechanism fostering equality of opportunity and 
thus redistribution of income. But other institu- 
tions in the society also exist to remedy one or 
more of these ills. When a university takes on bur- 
dens outside its mandate, it weakens its ability to 
perform some of its classic functions — which may 
not be bad, but which does point up the central 
problem of this symposium. Higher education is 
many things to many people: it is impossible to 
compile a self-consistent, yet complete, list of the 
goals of a university. 

The paper is organized as follows. Section I 
discusses recent technical developments in higher 
education administration. Section II, the heart of 
the paper, argues that for some of their activities 
faculty members and students should be considered 

* The auihor is grateful for the advice of James Jern* 
berg, Eugene Eidenbcrg. Robert Holt, and the partici- 
pants in the Conference on Public Service Outputs of 
Higher Education 



as acting independently, not as producing “out- 
puts” for their university. Furthermore, it is 
claimed that the analytical techniques for estimat- 
ing production functions of firms are inapplicable 
to universities. / .ter that. Section III outlines what 
is left of the public service outputs of higher edu- 
cation. Section IV looks at these outputs through 
the eyes of the four groups of constituents sug- 
gested by the organizers of this symposium. 

I, Recent developments, education finance 
and management 

In the last decade the field of higher education 
administration or management has been influenced 
by three important developments: 4 (1) the exten- 
sion of general public finance analysis to the costs 
and benefits of higher education; (2) the flourish- 
ing of the point of view that education can be con- 
sidered as developing human capital, as an invest- 
ment in making people more productive resources; 
and (3) PPBS (planning-prcgramming-budgeting 
systems) and all of the works and pomps associ- 
ated with this management device. 

The contribution of the economics of public 
finance to higher education administration has been 
to emphasize what is clear to most people — that 
the unfettered market cannot be expected to allo- 
cate education properly. Capita! market imperfec- 
tions make it difficult, if not impossible, for an in- 
dividual with no collateral but himself to borrow 
money on his future. Education of a student sup- 
posedly carries benefits foi others in the society as 
well, but these externalities tend to be ignored by 
the individual purchaser of education. Finally, and 
most importantly, the market can in certain circum- 
stances allocate goods and services efficiently, but it 
is blind to questions of fairness; as a society we 
may wish to subsidize some of our members other- 
wise without the wherewithal to finance their de- 
sires for higher education. Historically, America 
counteracted these deficiencies of the market by 
providing universal free elementary and secondary 
education and mammoth subsidies to higher edu- 
cation. 

The infusion into this situation of public fi- 
nance analysis and the economist's penchant for 
charging the beneficiary of a good for the good 
have led recently to increased use of the market. 
Where subsidies are desired this has meant moves 
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away from governmental support of the institutions 
of higher education and toward support of students 
deemed worthy of assistance by reason of their un- 
usual skills or financial deprivation. 5 

In the 1960s came apparent statistical justifi- 
cation for these subsidies; education viewed as the 
forging of a productive resource (people) seemed, 
overall, to be a worthwhile investment. 6 But over- 
all estimates of the productive value of education 
for the nation do not indicate whether incremental 
expenditures on education are well spent, nor do 
they uncover the relative effects of the various 
components of the educational process (teachers* 
brains and training, school buildings themselves, 
textbooks, native ability of the student, parental 
influence). 

However, attempts to go the next step, to esti- 
mate production functions for education — to de- 
fine and measure critical inputs and outputs of the 
educational process — have been unsophisticated 
in technique and disappointing in results. 7 

The disappointment is keenest where evalua- 
tions have not shown startling achievement in- 
creases for students in compensatory education 
programs. 8 Hope that America*s social ills could 
be overcome through educational efforts, such as 
Title I of 'he Elementary and Secondary Educa- 
tion Act of 1965 or the Head Start Programs of 
the Office of Economic Opportunity, must now 
rest on the conviction that our evaluative instru- 
ments are faulty or that the best in current prac- 
tice is not now present in these compensatory pro- 
grams, but could be introduced. 

Very recently there has been some indication 
that earlier assessments of the /ole of education 
in economic growth were exaggerated. Fast grow- 
ing industries no longer seem to have a relatively 
greater need for highly educated personnel: “Cur- 
rently, we have very little evidence that the pres- 
ent level of technological change needs to be sup- 
ported by even higher levels of education.** 0 

Significantly, these and other signs of weak- 
ness in the human capital argument for income re- 
distribution through education, have led recently 
to questioning and even renunciation of the educa- 
tion-as-investmcnt approach and a corresponding 
championing of education as consumption. Some 
lawyers and constitutional scholars are now arguing 
that test scores are irrelevant in making a case for 
improved education for the deprived. 10 If suburban 
students can luxuriate in small carpeted classrooms 
equipped with dazzling educational paraphernalia, 
how can one argue that their disadvantaged con- 
temporaries do not deserve the same, regardless of 
whether they affect educational outcome? And 



surely, demands for open enrollment in higher 
education grow, at least in part, from the percep- 
tion that college is where it*s at, never mind what 
happens in the future. 11 

So, despite some fanfare in the economics pro- 
fession, the human capital formulation has so far 
had little direct influence on public policy. 

Simultaneously, with the public finance and 
human capital developments, a new management 
tool, PPBS, has begun to be implemented in some 
institutions of higher learning. Ideally, a PPB sys- 
tem should involve an information system organ- 
ized to keep track of outputs rather than inputs, 
good analysis, and organizational arrangements 
which insure that policy is translated into action: 
That individuals acting in their own interest are 
furthering society*s interests. So far the great bulk 
of the work that has gone into developing PPB sys- 
tems has been in the first of these; that is, into 
sterile accounting schemes and computer cross- 
walks. Good analysis and organizational reform 
happen much less frequently. 

These developments — PPBS plus the eco- 
nomics of public finance and of human capital — 
have brought about the current output orientation 
in higher education administration. But the quest 
for the outputs of education is severely crippled by 
serious and imperfectly perceived (by many quest- 
ers) methodological and technical difficulties. The 
next section outlines these difficulties. 

II. The University As Non-Organlzatlon 

There are irreconcilable differences in this so- 
ciety as to what the outputs of higher education are . 
PPBS does not provide a theory for deciding what 
they are and how to measure them. Presumably 
economic analysis can provide the theoretical 
skeleton and mathematical stnlistics the empirical 
flesh to this elusive animal — educational output. 
But economic theory has to do with maximization 
(consciously or unconsciously) of known objec- 
tives. Where an objective or output is known (or 
felt) and an organizational entity is motivated to 
maximize it, economics provides a descriptive the- 
ory of how maximization will happen or prescrip- 
tive analysis as to how to get it to happen. That is, 
with existing analytical tools, economists can 
classify, measure, and maximize outputs when they 
are identified, and the incentive to maximize exists. 
But economics is not of much help in determining 
objectives or outputs and is misleading in situa- 
tions where objectives are not maximized, which 
is clearly the case in higher education where there 
are numerous competing viewpoints of what is 
going on. 
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A university is not a person or a firm; it is not 
the single-minded entity presumed in the use of 
management tools which have brought about this 
output orientation. No budget or plan of a college 
is compatible with a single, self-consistent set of 
preferences. 

What to Measure 

The notion of an output has implicit in it that 
an entity or mechanism is seeking to produce an 
output, indeed maximize it. Where there are con- 
flicting views or theories of what “is” and of what 
“should be,” there can be no agreement on what 
are relevant outputs. For it is a viewpoint or theory 
of the educational process which tells one what out- 
puts are, what should be measured. “Measurement 
without theory . . . ” 1 - is a delusion. Different views, 
different theories , of the educational process — that 
it produces humane citizens or productive cogs; pre- 
serves the wisdom of the past; guarantees a tech- 
nologically advanced society; furnishes a safe and 
fruitful base from which to effect societal reform 
or revolution; provides an enjoyable interlude be- 
tween childhood and work-a-day-ism — these dif- 
ferent views suggest different outputs, and who is 
to say that one view is “right”? Compiling long 
lists of “outputs” is of little usefulness in the ab- 
sence of behavioral theory relating inputs to the 
outputs, 

Howto Measure 

The obvious response to the foregoing para- 
graph is that one can always list all the outputs 
suggested by all Ike competing theories and then 
proceed to measure the degree of attainment of 
each. Here, the standard procedure is regression 
analysis relating an output on the left side of the 
equation to a host of inputs on the right, the co- 
efficients of the input variable presumably indicat- 
ing their relative importance. But even that will 
not do. n It will do as a collage, an encyclopedic 
description which might be distributed as an ap- 
pendix to the college catalog. But it will not do as 
a guide to public policy. For what the legislator, 
university administrator, or educational reformer 
wants to know is what will happen if a conscious, 
concerted attempt is made to try to get more of a 
particular output. Measuring what has been hap- 
pening in a university where many outputs are be- 
ing produced and no one or no cluster is being 
maximized will not yield information on efficient 
allocation or even on what would happen if an at- 
tempt were made to get more of a partiadar output . 

The production function - regression analysis 
approach of the cost-benefit analyst is borrowed 



from the economics of the firm, where the profit- 
maximization axiom provides a rationale for claim- 
ing that the regression equation yields a Jocus of 
efficient combinations of inputs. Where there are 
joint outputs in a non-maximizing environment, 
there can be no presumption that input-output cal- 
culations yield either efficient patterns or patterns 
that are even replicable in a different time or 
school, where different preferences may prevail. 

Lacking powerful behavioral theory on the ac- 
tions of academic bureaucrats and political theory 
to predict the outcome of clashes of competing 
claims, we are in no position now to measure all 
the outputs of a university. Existing political analy- 
sis on resolving conflict is of little value in this 
instance. Prevailing political theory, consciously or 
unconsciously patterned after the invisible hand 
notion of economics, has aspired to be both de- 
scriptive and prescriptive — ratifying outcomes of 
the political process on the grounds that competing 
claims are, in fact, settled in the political “market 
place.” But dissatisfaction is very great with this 
characterization of political processes (in govern- 
ment o • universities ). 14 

Not only are there conflicting views on goals, 
but running a university may be a contradiction in 
terms. At any rate, it is a far cry from running a 
firm. “The main pattern of academic organization 
is associational rather than bureaucratic. It docs 
not stress ‘line 1 authority, since basically all full 
faculty members are formal equals .... Indeed, 
relative freedom from the authority of organiza- 
tional ‘superiors’ is one of the main aspects of 
academic freedom .” 15 Academic organization is, 
then, the institutionalized antithesis of the firm or 
government bureau, where superiors control the 
actions of subordinates. T he fact that a university 
is not a unit is what allows diversity in teaching 
and research . 10 

No wonder then, that there has been little 
progress in measuring the outputs of universities. 
Existing techniques were designed for organiza- 
tions with an incentive to produce efficiently an 
agreed-on product. In contrast, a university is in 
many ways, a non-organization, where there is no 
agreement on its product, the independence of the 
individual faculty member is valued highly, and 
there is surely no inherent goad to maximize. In- 
deed, a mechanism for maximizing some com- 
bination of outputs may be inconsistent with the 
idea of a university. A university is designed not 
for reconciling conflicts, but for permitting them . 17 

For many purposes, the activities of individual 
faculty members should not be linked to the uni- 
versity. This concept narrows the range of the uni- 
versity's involvement in public sc r vice. 
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III, Public Service Outputs 

Four output categories were suggested for this 
symposium: undergraduate education, graduate 

education, research, and social involvement. In 
order to distinguish from the outputs in the first 
three categories, outputs of public service or out- 
puts of social involvement activities of an institu- 
tion of higher education are defined here as non- 
research oenefils accruing to persons other than 
students and employees. As might be inferred from 
the two previous sections, ideological convictions 
as to the nature of a university, combined with 
practical assessment that an institution of higher 
learning is ill-suited to carry out many social mis- 
sions, lead me to a narrow definition of what 
should properly be considered public service out- 
puts . 17 

The guiding notion in what follows, then, is 
that public service activities of faculty members 
are not the concern of the administration of a uni- 
versity. Faculty members should be free to dabble 
in the design and operation of programs combatting 
the problems of crime, pollution, race, and poverty 
and be paid to do so by anyone willing to believe 
that professors are good doers. Correspondingly, 
universities needn't measure the “outputs” thus 
generated, nor pay faculty members for time thus 
spent. There remain some public service activities 
of the university, per se t and these will be dis- 
cussed shortly. 

In every case tficy grow out of the teaching or 
research missions of the university. They come 
about, jointly produced with teaching or research, 
as external benefits (or losses!) spilling over to 
nonstudents from the educational process. 

Public service outputs seem to be of three 
kinds: 

1. Direct services to Individual* 

Direct services to individuals can be divided 
between those for which the university is reim- 
bursed and those for which it is not. The distinc- 
tion is crucial since, in the former case, measure- 
ment of benefits is made unnecessary (or at least 
much easier to accomplish) by market valuation. 
Interestingly, one such activity, often listed as a 
public service, is university extension. This is in- 
appropriate when extension is simply courses of- 
fered to students in the evening or by mail. Ex- 
tension is typically sclf-rupporting , 18 all the more 
reason for listing it as undergraduate or graduate 
education, not under the public service rubric with 
attendant connotations of largesse on the part of 
the university. 



Another type of "public service,” which like 
extension does not belong in this category, is gov- 
ernment-subsidized training for particular profes- 
sions. Thus, universities abound with federally fi- 
nanced programs designed to produce specialists 
in tranportation, urban planning, systems analysis, 
etc. These should be priced at cost and listed as 
undergraduate or graduate education. Presumably 
the student, not society, is the recipient of the 
benefits of such a program, for. if when he leaves 
the university he is b net at what he was trained 
for than when he came, he will be paid more. (If 
a buyer — say his former employer — is not will- 
ing to pay him more, someone else will .) 19 

But there is a class of direct services to non- 
students which can properly be considered public 
service. These are activities which flow necessarily 
from teaching and research. Hospitals and dairy 
stores are examples. The problem here is not out- 
put measurement, but price determination, the 
difficulty being that the service is produced jointly 
along with teaching and research, and it is impos- 
sible to attribute costs to the three . 20 

The difficult problems under this first cate- 
gory arise when the university is not reimbursed 
for a service. The most important example is the 
filtering or credentialing process which universities 
perform for employers. Students with the stamina 
to make it through college obviously are identified 
as of sonic value apart from what they know. This 
function is akin to that of discovering talent, which 
Theodore Schultz has elevated to the position of 
one of three major functions of higher education, 
along with instruction and research .- 1 The value 
of this service is. at least, potentially measurable. 
One might begin, for example, by noting the dif- 
ference between what employers arc wilting to pay 
two individuals, one >vith and one without college 
education, whose performances on standardized 
achievement and aptitude tests are identical. 
Though sorting or “discovering talent” may be 
valued highly by some, it is not universally re- 
garded as a good. Town-gown physical confron- 
tations and demands for ungraded classes are in- 
dications that not everyone appreciates being 
pigeon-holed by the academic process. 

A final item in this category of non-rcimbursed 
services is public goods, for example, radio and 
television broadcasting .- 2 Again, these are often 
jointly produced with instruction, and it is diffi- 
cult to measure their benefits. In the absence of 
scrambling devices on owners* sets or some other 
exclusionary mechanism measuring the value of 
this service is subject to all the difficulties of pric- 
ing any public good, where a recipient has an in- 
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ccntive to understate (to the producer of the serv- 
ice) the value he places on it.- s 

2, Income Transfer* 

Traditionally, higher education in this country 
has been regarded as an equalizer o? opportunity. 
No doubt many Americans feel that the Horatio 
Alger myth justifies the fact that very few institu- 
tions of higher learning charge students full in- 
structional costs. Philanthropies and governments 
subsidize education, not only in recognition that 
research costs should not properly be borne by 
students, not only because of external benefits to 
society, not only because the money market for 
student loans is inefficient, but also because of a 
belief that low tuition subsidizes the economically 
disadvantaged. This clearly is an empirical ques- 
tion, one on which little work had been done until 
very recently. Recent findings by VV. Lee Hansen 
and Burton A. Weisbrod on the distribution of edu- 
cation subsidies in higher education in California 
are shocking.- 4 Far from establishing the generosity 
of the well-heeled, it appears that, on balance, 
educational subsidies rise for successively higher 
income groups. (The university system is costlier 
than the state colleges which are costlier than the 
junior colleges which are costlier than not educat- 
ing a person at all. But the average income of 
Berkeleyites is greater than the average income of 
San Francisco State students, etc.) Hansen and 
Weisbrod discovered that the situation in Wiscon- 
sin is less discomforting, emphasizing the great need 
to perform the relatively straightforward task of 
measuring educational subsidies by income group. 
This task should be extended to include the meas- 
urement of subsidies by ability level as well. For 
unless there arc great externalities associated with 
educating the most capable students (which l 
doubt), then the widespread practice of awarding 
scholarships to bright students may be but another 
bit of income redistribution from the less well off 
to the better off in the society. This is not only 
because such students come disproportionately from 
higher income groups, but also because being both 
talented and educated, they will tend to be highly 
paid fVi the future. 

Note that dropping tuition rates to zeto for 
all is hardly a solution, since most of the cost of 
higher education is the wages foregone while in 
colleges* It is certainly plausible that at zero tui- 
tion, these costs are more prohibitive to the poor 
student than to the rich on c.-° 



3. External or spillover benefits from students 
to others In the Society 

In 1966 Theodore Schultz had called for some- 
one to measure the income redistribution effects of 
higher education; that work is under way.- 7 Now 
the great need is to measure externalities. It is be- 
ginning to appear that the most overrated argument 
for public subsidies of higher education is that 
made on the grounds that there are externalities 
or spillovers from an individual student’s education 
which benefit ls all. The only serious study of this 
question is Burton Weisbrod's 1963 monograph. 
Spillover of Public Education Costs and Benefits.-* 
But this was concerned primarily with pre-college 
education and by 1969 Weisbrod (with W. Lee 
Hansen) was saying, “Our apparent skepticism 
about either the existence or significance of some 
of the widely discussed external benefits from high- 
er education stems principally from the absence 
of any substantial body of evidence in support of 
them." 20 Here, then, the problem is not one of 
devising operational measures of outputs, but in 
finding empirical evidence that they exist. Indeed, 
when they do, they may be negative. :m Categories 
for consideration of externalities have been worked 
out by Burton Weisbrod 41 and can be accepted 
here. (See table.) 



Public Service Outputs 

Direct Services to Individuals 

Reimbursed 

Non-refmbursed 

Income Transfers 

To economically disadvantaged 
To academically adept 

External Benefits to Non-students 

Residence-related 
Employment-related 
Society in general 



IV. Constituencies and Their Perspectives 

This section sorts out the above services, trans- 
fers, and benefits in terms of the categories defined 
by the organizers of this conference. Four con- 
stituencies were proposed: the public^ private in- 
terests, faculty, ;jnd students. 
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The Public 

Higher education’s services to the public are: 

1. Redistributor of income (scholarships) 

2. Provider of public goods (broadcasting of 
faculty lectures) 

3. Furnisher of benefits which spill-over from 
the instruction process (I do not know what 
these are.) 

4. Market for services (Training for profes- 
sions deemed worthy of subsidies.) 

The need to price at cost and/or to keep track 
of who is getting what, is great. Many higher edu- 
cation activities are billed as benefiting “the pub- 
lic,” when, in fact, they merely represent the ef- 
fective lobbying of special interests, including uni- 
versity faculty/ 2 

The Private Sector 

The last sentence is, of course, relevant here. 
For the private sector the university is: 

1. Provider of training services (often not a 
“public service”) 

2. Seller of services produced jointly with in- 
struction or research 

3, Seller of “monuments”™ 

4, Sifter of talent 

The last function is the most difficult to handle, 
since it is difficult to set a price for it. 

Faculty 

From the standpoint of public service, this 
paper’s main argument concerning faculty is that 
they ought to deal as independent agents, not as 
part of a university organization. 

Students 

To students the university’s public services are: 

1. Provider of income transfers 

2. Provider of services 

Some students are recipients of income trans- 
fers from the rest c r society. Fn the absence of 
evidence of external benefits from education to 
the public, it is difficult to make a case for such 
a transfer to all students. Students as members of 
the public ought, of course, to be eligible recipients 
of such direct services as the university provides, 
as, for example, health care. 

V. Conclusion 

Much accommodation of special interests has 
masqueraded as public service. Faculty members 
doing public service, which is independent from 
teaching and research, should do so as independent 



agents. University services should be priced at 
marginal costs in the interests of both efficiency 
and equity. Economic analysis, which has fostered 
the output orientation in higher education, is not 
appropriate for finding or measuring educational 
outputs. Universities are neither the firms of neo- 
classical economics, nor the bureaucracies of the 
Weberian tradition in sociology. Not only are the 
measurement techniques inappropriate, but pre- 
cisely in those areas (public service) where they 
are inappropriate, they are unnecessary, since the 
university qua university is not producing those 
outputs — individuals. 
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"Bated on (he numbers of baccalaureates who go on to re- 
ceive doctorates, there are cons Iderable quality differences 
among undergraduate program a.” 

"A curriculum . . . reflect* the conceptual structures of the 
era In which the IHL operatet. To tome extent tuch changet 
may be trantmllled directly to atudente. Mora Importantly, 
tor the ttudent, contact with the university environment may 
partially leglthnlte the personal Innovations which he or the 
undertakes." 

"In spite of the grumblings over Inequities of the 'publish or 
perish' system, scholarly output continues to be one of the 
few explicit measures by which facully performance Is 

fudged." 



"Support lot educational research doubled between I960 
and 196$. Our tallure to provide an acceptable understand • 
Ing of what education has done and la doing for the young 
people of the nation Is probably related to the fact that only 
.22 per cent of the total budget for education was spent on 
educational research." 

"Consider the economic rewards which How to a recipient 
of a good education." 
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The Outputs of Undergraduate Education 

ROBB1N R. HOUGH 



Of apple picking: \ am overtired 
Of the great harvest I myself desired. 

There were ten thousand thousand fruit to touch. 

Cherish in hand, lift down, and not let fall. 

Tor all 

That struck the earth, 

No matter if not bruised or spiked with stubble. 

Went surely to the cidcr-appfc heap 
A ; of no worth. 

“After Apple Prel e" Kobert Frost. 



The literature on the economics of higher edu- 
cation docs not dwell long on the institution of 
higher learning (IHL) as a decision-making and 
goal-seeking enterprise. 1 1 has been most fashion- 
able to treat IHLs as black boxes into which inputs 
arc poured and from which outputs flow. 

It is a primary assumption of this paper that 
the IHLs of the nation arc decision-makers in a 
fashion which makes them directly comparable to 
the firms of micro-cconomic theory. Like firms and 
consumers they are decision-making institutions 
possessing clearly specifiable goals and instruments 
for attaining th.*c goals. They* too. operate in 
environments which constrain them and to which 
they react in ways which are predictable. 

It is assumed that IHLs are institutions which, 
for a range of reasons, are operated by their fac- 
ulties. but are subject to considerable constraints 
by administrative, student, trustee, and. in the 
case of public institutions, legislative constitu- 
encies. In consequence, the “behavior** of an IHL 
will be determined as a resolution of the some- 
times conflicting goals of these several constitu- 
encies. 

The purpose of the paper is twofold: (I) to 
define operational!) some outputs of undergradu- 
ate education and (2) to explore briefly a model 
of IHL behavior which may throw light on im- 
portant relations between the outputs defined and 
the inputs which give rise to them. In all, the 
paper is based on a commitment to cast IHLs into 
the mold of a different perspective. It is a paper 
to be mused rather than digested for immediate 
action. 

A Faculty View 

From the point of view of a college faculty, 
treated as a unit, three categories of output will 



receive consideration. These categories arc educa- 
tional outputs, change* and outputs which con- 
tribute to professional advancement and recogni- 
tion. 

Reliable Measure Lacking 

A primary problem in the study of under- 
graduate education is the lack of a consistent and 
reliable measure of the quality of IHL output. 
Ideally, such a measure would relate to the long- 
run ocrformance of the graduate beyond the cam- 
pus. The development of such a measure will be 
discussed in the next section of this paper. Lack- 
ing such a measure, a means for making limited 
intcr-institutional comparisons is available by dis- 
tinguishing between three categories of student 
outcome. Dropouts are. of course, students who 
begin but never complete work on a baccalaureate 
degree. It is useful to distinguish between two 
l>pes of students who do go on to complete their 
degree work. The general idea involves distinguish- 
ing between those students who meet the formal 
requirements for a degree and those who are sin- 
gled out by the faculty as meriting special care, 
attention, and or commendation. The latter will 
be called "apples’* throughout the remainder of the 
paper. 

A> applied to a given IHL. the term "apples** 
will be defined as those baccalaureates from the 
IHL who go on to receive doctorates and to those 
doctorates produced by the IHL. Several justifica- 
tions underlie the consideration of these specific 
groups as they relate to institutional quality. First* 
in an educational setting* the purview of a given 
faculty is limited, and special attention can be paid 
to a limited number of students, be they graduate or 
undergraduate students. Secondly, despite objective 
testing, the evaluation of student performance is 
still a strongly subjective matter. In the case of 
“apples.** the judgments of an undergraduate fac- 
ulty have been confirmed fin some scn<e) by the 
judgments of a graduate faculty. Finally* the meas- 
ure allows the somewhat cvcnhanded con ipa risen 
of undergraduate with graduate institutions In 
connection with this final point, preliminary’ work 
with the measure suggests that the measure pro- 
duces a rank-ordering of undergraduate institu- 
tions which is consistent with the folklore on aca- 
demic quality. An examination of this measure in 
comparison to other measures is under way. 
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Based on the numbers of baccalaureates who 
go on to receive doctorates, there are considerable 
quality differences among undergraduate programs. 
On average, about .4 "apples” were produced be- 
tween 1958 and 1966 for each degree granted in 
1967 by 75 of the largest undergraduate “apple" 
producers. However, the ratio of apples to de- 
grees ranged from a low of .08 to a high of 2.63 
(National Academy of Sciences, 1967, and HEW, 
1967). The twenty largest doctorate-producing 
institutions over (he period had a total combined 
undergraduate enrollment of 288,477. They pro- 
duced a total of 21,484 baccalaureates who went 
on to receive their douorates during the period. 
The four largest doctorate producers had a com- 
bined undergraduate enrollment of 69.744 and 
produced 6,233 baccalaureates who received doc- 
torates during the period. The twenty institutions 
with (he highest ratios of "apples" to degrees had 
combined undergraduate enrollments of 56.969 In 
1967 and produced 11,921 baccalaureates who 
went on to receive doctorates. The four with the 
highest taltos had 8.337 undergraduates and pro- 
duced 3,253 who went on to receive doctorates. 

The largest producers of baccalaureates who 
later tcceive doctorates and those producers with 
the highest ratios of undergraduate "apples" to 
baccalaureate degrees are quite different. The larg- 
est producers include a number of public institu- 
tions. The institutions with high ratios are private, 
relatively small, have low student-faculty ratios, 
demand high admissions scores, and pay high fac- 
ulty salaries. 

The relations between the degree recipients, 
“apples," and dropouts will be discussed in a later 
section of the paper. Let us now turn to the dis- 
cussion of a second type of output. 

Centers el Change 

Though they may be so only jn the minds of 
their faculties. IHLs have often been thought of 
as centers of change. In an attempt to consider the 
full range of 1HL outputs, some attention should 
be given to the role these institutions play in the 
diffusion of technical and cultural changes. 

A curriculum, in part, reflects the conceptual 
structures of the era in which the 1HL operates. To 
some extent such changes may be transmitted 
directly to students. More importantly, for the 
student, contact with the university evironment 
may partially legitimize the personal innovations 
which he ot she undcrtaV.es. It may be that major 
cultural innovations are transmitted in a less direct 
fashion. In such a view, the IHL may be thought 
of as the force for stimulahng interregional flows 



of students, faculty, and information through which 
innovations are diffused. It may prove possible to 
study :he impact of these flows by examining the 
sprer.d of particular innovations, such as under- 
ground movies, coffee houses, fashions, and the 
like. 

In spite of grumblings over the inequities of 
the "publish or perish" system, scholarly output 
continues to be one of the few explicit measures 
by which faculty performance is allegedly judged. 
Presumably, it. will continue to be a useful index. 

Output as measured by books, articles, and the 
like does not adequately capture what might be 
called the storage and dissemination functions per- 
fotmed by faculty. Investigators in the informa- 
tion sciences have long been aware of the numer- 
ous informal channels of communication which con- 
nect the researchers in disparate geographical cen- 
ters (Garvey and Griffith, 1964). The public af- 
fairs and consultation activities of faculty link (he 
formal and informal communications channels to 
society at large. By participation in these activi- 
ties and by teaching, the faculties perform com- 
munication and the dissemination functions. A 
study of the extent, variety, and dollar volume of 
faculty consulting might help in the understanding 
of these functions. 

As mentioned above, much depends on the 
development of an adequate measure of teaching 
effectiveness both at the individual faculty level 
and at the institutional level. From no point of view 
is the question more important than from that of 
the student. 

A dludtnt Vl«w 

This section will be concerned with the ques- 
tions of intellectual value-added and economic re- 
wards. Topics will be treated In turn, though there 
are important overlaps between them. 

Support for educational research doubled be- 
tween I960 and 1965. Our failure to provide an 
acceptable understanding of what education has 
done and is doing for the young people of the 
nation Is probably related to the fact that only .22 
per cent of the total budget for education was 
spent on educational research. While it may help 
somewhat to understand the determinants of "ap- 
ples" production, sv its are beginning to ask 
for much more spccit v justification of the curricu- 
lar tasks they are asked to carry out. In order to 
proside these justifications, it would seem that 
measures of intellectual value-added, more prom- 
ising than those in use. must be developed. The 
problem is well stated by Bennett. 
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To Justify transferrins the IQ to fields of human 
endeavor other than the psychological laboratory it 
would be necessary to show how it co* related not only 
with scholastic perfoimance but also with practical 
life achievement. A ^reat deal of work has been done 
in this field, and it is generally agreed that the 
quantity measured in an intelligence test is strongly 
correlated with scholastic achievement The evt- 
dence of co-relation with life achievement is very 
much weaker This can be accounted for if we as* 
sumc that intelligence is an organized “something'* in 
man that determines his ability to perform opera* 
lions of a certain kind, but that there are other 
systems that play a less well-defined but no less 
important role (Bennett, 1969), 

There is a line of research which it might be of 
some use to consider in pursuing the quantification 
of these particular educational outputs. In the 
writer's modeling of the university classroom, the 
distinction between a student's information-process- 
ing capacity and memory capacity became quite 
useful (Hough, 1968). For those comfortable with 
the analogy between a computer and a human, 
the distinction between the central processor ca- 
pacity of a computer and its memory capacity is a 
usef jl analogy. A recent discussion by Cronbach 
specifies a similar distinction, though he specifies 
the use of memory capacity rather than its size, 

On the scientific side, it is vita? to break away 
from such l*rms as “intelligence'' and “learning abil- 
ity." There ; s a spectrum of performance ranging 
from crystallised. over-learned routines to fluid in- 
formation procsssing, often referred to as g. Fluid 
aSHty is measintd in tests like the maze, the matrix 
of figure analogs, block design, and embedded fig- 
ures. Jensen’s Level II abilities involve it. Crystallired 
abilities are dis*erse and specific: spelling of -gn 
words, handling of subjunctive clauses, etc. In schools 
as they now are, success is best predicted by faking 
an inventory of relevant crystallized abilities with 
which the student starts the year. The verbal “in- 
itl'igence" test succeeds as a predictor primarily be- 
cause it reflects concrete achievement. A child with 
average fluid ability and low costallired ability is 
likely to do poorly; we have never succeeded in de- 
vising a mass educational program in whkh such a 
child is lik«b to achieve average success. Ar.aljtk 
ability should be a resource on whkh education 
builds, and m of now it is not (Cronbach. 1969). 

Perhaps even more so than in elementary and 
secondary education there is a tendency for the 
undergraduate classroom to place reliance on a 
stock of crystallized abilities whkh the student is 
supposed to possess or be aWe to acquire qukkly. 
ff the student has not acquired or has limited 
capacity to store crystallized abilities, difficulty will 
surely aHse. There is indeed some e\idenee that 
undergraduates with relatively low crystallized 
abilities have a relatively high dropout rate, even 
when their fluid processing abilities are extraordi- 
narily high. 

To the extent that students differ considerably 
in their pc ssession of these abilities, their responses 
to partkular environment* for learning may differ 



significantly. Som: rather too casual evidence from 
the writer's classroom experience tends to link rela- 
tively strong endowments in cither direction with 
distinctly different learning styles. Students who 
display relatively high levels of fluid processing 
ability tend to react to classroom situations by 
describing themselves as “bored.” Their colleagues 
with relalively high endowments of crystallized 
skills tend to describe themselves as “tired ’ under 
similar circumstances. A belief in comparative ad- 
vantage has led the writer to conclude that ttnse 
with fluid processing ability seek “concepts” which 
they can apply and are “bored” with an environ- 
ment in which they are asked to listen to numerous 
facts. Those students with crystallized abilities try 
to remember all that is being put forth, and they 
are seldom able to keep up with the rale at which 
the environment of a classroom produces informa- 
tion. 

Careful study and definition of fluid processing 
or crystallized abilities, as they relate to specific 
courses of study, would do much to clear the way 
for more adequate measurement in these areas. For 
example, the process of “doing' economics may 
be caricatured as accepting inputs, processing those 
inputs, and writing articles for the literature on 
economics. To the economist, the inputs are ob- 
servations of behavior and measures on the ob- 
served behavior. Within the “black box," models 
of the observed behavior are constructed and de- 
rived hypotheses tested against the measures. Can 
the crystallized skills necessary to the doin$ of 
economics be inferred from the output of the 
economist? Are the concepts u^ed to wtite about 
economk behavior in one-to-one correspondence 
with those crystallired skills? What level or range 
of fluid abilities is required? What special subset 
of analytic concepts could be useful for students 
with a relatively low capacity for retaining crystal- 
lized skills? 

Skinner seemed to have been on the right track 
when he wrote: 

A much more promising approach is to took 
at the student** behavior — it* behavior from *hkh 
mentalistk Mate* and processes are inferred and 
which they %o inadequately describe and explain. The 
bask question, in it* crudest form, i* this: IVtar 
do *r nan/ the tf*det\l to do ai a tetHlt of herinft 
hetn Uvfthi 1 * tit is no answer to cite examination* 
he is to pas*. for they are only sample* of hi* be- 
havior. and no matter how reliable they may be. 
they are. we hope, very small sample* indeed of 
what he will actually learn ) To say that me want 
the student to "behave like a scientist" is on the 
right track, but it n only a start. For how doe* a 
scientist behave? The answer will be nothing les* 
than an epistemology, a theory of scientific knowl- 
edge. It mint in fact be more: W’e need an empiri- 
cal description of the behavior of the scientist at 
work, rfi all it* myriad form* (Skinner. 196$). 
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Consider the economic rewards which flow to 
the recipient c. f a good education. Considerable at- 
tention has been paid to education as investment 
in human capital. Most of that has been focused 
on the returns to the individual as a participant in 
and contributor to the process of economic growth. 
Much excellent work has been done in that area. 
It docs nol seem appropriate to review that work 
here except perhaps to note that it would be highly 
useful to continue studies of the type undertaken by 
Hun* in which the focus is on the factors which 
contribute to individual differences in the rates of 
return (Hunt. 196>). There are two labeling or 
sorting processes accompanying undergraduate edu- 
cation which deserve further attention. Graduates 
arc labeled as a product of their alma mater* and 
are given degrees which correspond to particular 
compartments within the institutions they attended. 
It is not clear that the labeling which goes on in 
the context of undergraduate education is an ur»- 
qualified economic benefit from that process. While 
it may prove to be of short-run use to both the 
graduate and his employer, the existence of the 
strong occupational stereotypes pointed out by 
Bcardslcc aril O'Dowd may serve in the long run 
to reduce subslanlially the individual's perceived 
mobility and. thus, his actual mobility (llcardslcc 
and ODowd 1962). 

There is an additional question which must be 
raised front the student's point of view. 

Though SchuMr recc^nkc* that a part of ihc 
education given and received is really a consumer 
good, he does not separate that patt from the total 
capital formation by education, ft might he argiieJ 
that even though there i* *oroe education that 0 es- 
semiallv used as a consumer eood. it could skid 
productive service*. ju*t a* the education intended pri- 
marily to form human capital doe*. Therefore it. 
too. should be added to the capital stock created hy 
education. Thi* addition would be hard to 
however, in the face of the well-known difference* 
in the returns from different kind* of education 
(Eckhau*. 1962). 

Is Eckhaus correct, or do we simply fail to 
account for a substantial economic benefit which, 
in fact, might be measured? 

Consider consumer goods and an individual's 
time as factors of production in a production pro- 
cess. the output of whkh is the ” . . . restoration 
of a depleting psxchk capital." (Utility?) Boulding 
suggests that there are two dimensions to such a 
product: 

Thus, f have afft*d that we po to a concert in 
order to restore a p*ychk condition whkh mipht be 
catted ’'fmt havint pone to a conceri " whkh, once 
established, tend* to depreciate. WT>en it depreciate* 
beyond a certain point, w* go to another ccOcttf in 
order to restore it. If it depreciate* rapkHr, at po 
to a M of concert*; if h depreciate* sfernty. wt po 
to few . » . , 



Further interesting Problem* are raised by the 
demand for variety. We certainly do not want a 
constant state to be maintained; we want fluctuations 
in that state. Olherwbc there would be no demand 
for variety in food, for variety in scene, as in travel, 
for variety in social contact, and so on (Boulding. 
1966). 

Changes in the wage rate of the individual (be., 
the price of his time), relative to the prices of the 
other factors used in the "psychic production func- 
tion" will affect their relative employment in the 
restoration of psychic capital. The higher the price 
of consumer-good factors, the greater the tendency 
to employ "do-it-yourself methods to restore 
psychic capital. Similarly, the higher the wage rate 
of the individual, the greater the employment of 
consumption goods. Mincer has studied the in- 
come elasticity of demand for domestic help us- 
ing a similar formulation (Mincer. 1963). Thu*, 
family members and domestic help were viewed as 
factors of production in the household. The failure 
to include the female wage rate as an opportunity 
cost variable resulted in biases in the income vari- 
able of more than 50 per cent. The studies of the 
role of education to date have simply concerned 
themsetves with the effect of education on the 
wage rate. If the formulation above is considered, 
the acquisition of crystaiiitcd abilities and the im- 
provement of fluid processing abilities may be 
seen to affect both the w age rate and the production 
function itself. The effect of certain skill acqui- 
sitions on the production function may more than 
offset the increase in consumption brought about 
by the higher wage rale returned to the skills. In- 
deed. this "busman's holiday" effect may. in part, 
be responsible for the observed relationship be- 
tween savings and income. 

A Public View 

An enumeration of the outputs of higher edu- 
cation whkh dominate the publk view these days 
will be foregone on the grounds that these eddks 
in Ihc serene pods of academe are but short run 
costs of long-run progress. Hopefully, someone can 
supply historkal justifkation for such a view. 

The economkx of education resemble nothing 
so closely as the economics of a world fair. That Is 
to say. it is very difficult to make ends meet J n 
the day-to-day operation of the I ML. but benefits 
front its existence are reaped much is they arc 
reaped by every merchant within miles of a fair 
site. 

The gathering together of the human resources 
rvxessarv to produce a particular type of under- 
graduate educaikm makes possible the presenta- 
tion of a number of other tipes of program. Among 
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these would be institutes, seminars, training pro- 
grams. and the like. The teaching, administrative* 
and technical personnel of IHLs often take diffi- 
cult and time-consuming tasks as community lead- 
ers and consultants. The specialized physical fa- 
cilities of these institutions are often made avail- 
able for publie use, A part of the cultural impact 
of an IHL can be seen to operate in the areas of 
audience development and spectator sports. It is 
well to recall the observations of Carl Kaysen: 

(U Since iiwiilutionv of hipher education pro* 
duce multiple products with partly overlapping and 

f >ard>’ different turn, it h important for them to 
earn what their product mix is who in fact are the 
beneficiaries of particular products, and. so far as 
possible, what are the separate vests of producing 
each output. 

(2) Some beneficiaries should and may be ex- 
pected to pay the "full costs" of the particular out* 
puts they use. anJ. in fact, prices to them may ap- 
propriety be higher in relation to directly assign* 
able costs than for others. Some t>pes of output 
benefit the community as a whole so much and par- 
ticular individuals so litde, as to make it both inap- 
propriate and infeasible to expect to charge them on 
a cost basis. Thus, a combination of intcr product 
price discrimination and a flow of receipts not con- 
tingent on the "sale" of a particular sort of output 
ii indicated fKa>seft. I960). 

Most of the products above are individually 
identifiable, and at least something like full-cost 
pricing is employed bv many IHLs. 

A number of effects will identify the IHL as a 
producer of externalities. The costs for the pro- 
duction of these particular benefits are, of course, 
not easily assignable. Some of them may be meas- 
urable. 

It is of considerable importance in those times 
that an IIIL is a pollution-free institution. Further, 
to the extent that many IHLs are desirable neigh- 
bors. they may have a considerable impact on the 
property values in the area surrounding their cam- 
puses. The techniques utilized in the study of land 
value gradients around city parks might be em- 
ployed in order to e<limatc «his particular range of 
outputs. In a similar fashion, some in a community 
may benefit at the expense of others, insofar as 
lilts attract a relatively high-ircome population to 
the area surrounding them and simultaneously drive 
away low-incon>c residents. Again, local com- 
munities may benefit from the considerable stimu- 
lus of the spending of faculty and students as well 
as from higher income and properly tax revenues 

To some extent, earning differentials among in- 
dividuals who are going to college and those who 
are not arc based on ability. The provision of a 
draft sanctuary of IHLs to an extent which is es- 
timable reduces the costs of the military draft. 
There also seerm to Le at leaxt a modest tendency 
for university enrollments to be counter -cyclical in 



that young folks who might otherwise be uncm- 
[ loyed utilize slack periods in the economy to in- 
crease their education. The opiion to continue in 
the educational system slows the rale of entry of a 
given cohort into the labor force and disperses the 
colic rt in ways which may be advantageous to so- 
ciety. 

Finally, the presence of a faculty usually exerts 
a stroi g influence for improvement in a com- 
munity**- school system. It may further serve as an 
attraction to light pollution-free industrial enter- 
prises. In both cases, there arc gains which arc 
clear in concept to the public. 

A Private Industry Viow 

As with die public outputs, there is a range 
of externalities which relates the IHL to the pri- 
vate sector. Insofar as an IHL community is a rela- 
tively attractive community, firms may find it easier 
to recruit employees to live in the community. 
IHLs make available n large pool of part-time 
labor of an unusually high quality. In a similar 
vein, it might be expected that the sorting and 
grading process which a given institution performs 
at a point in time 5s reliably related to the job 
which it docs at a later point in tine. Thus. IHLs 
become reliable sources for employees of a given 
quality. 

Also, as with public outputs, there arc many 
join products which arc probably paid for at some- 
thing close to their “full costs.*' Faculty consultants 
are I ired. personnel are asked to serve, facilities 
are hired, and the usual range of curriculum-re- 
lated programs is made available to employers. 

The IHL At a DecItlorvMakIng Institution 

The marketplace in which IHLs operate nay 
now be defined in terms of the supply of and the 
demand for degrees. That is. the outputs discussed 
above point to the role of the educational institu- 
tion as a certifying, or legitimizing, institution. The 
degrees, course certificates, consultations, leader- 
ship rotes, and labeling arc all subject to the cer- 
tification or legitimization interpretation. 

The 'firm/* then, might be said to produce 
in a market in which degrees arc suppli and de- 
manded. In the sense that '‘profits** earned may be 
“plowed back * to improve the “quality** of produc- 
tion or increase institutionally supported research, 
these firms may be said to maximize profits. 

As in analyses of the industrial marketplace, the 
analyst may be concerned with questions of (I) 
the degree to which the profit-maximizing model 
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“explains” ihe behavior of IHLs, (2) the structure 
and performance of (he industry, and. finally, (3) 
the policy implications of the model. The remain- 
der of this paper will examine some preliminary 
findings which bear on these questions. 

Educational "production functions" arc com- 
monly used to relate characteristics of incoming 
students to the characteristics of outgoing students. 
The depiction of the university as a certifying and 
legitimizing institution suggests that there is a more 
direct analogy to the firm of micro-cconomic 
theory. 

In the sense in which it will be used here, the 
“production function" of the IHL relates student 
and faculty Inputs to degree-holder (or certifica- 
tion) outputs, The "technology" under considera- 
tion may be thought of as determined by (I) ad- 
missions standards. (2) curriculum requirements. 

(3) the “technology" of classroom teaching, and 

(4) MIL degree requirements. Significant changes 
in any of these four factors may be thought of as 
engendering shifts in or distortions of the entire 
production surface. 

Assume that the primary outputs of the in- 
structional process arc degrees and that students 
and faculty are the only required inputs. The re- 
sults of fitting a Cobb-Douglas-lyp? function to 
75 of the 100 largest producers of baccalaureates 
who went on to receive doctorates arc found in 
Table I. In that table, the dependent variable is 
Ihe number of degrees produced in 1967. variable 
01 is the total enrollment in 1967. and variable 02 
is the number of full-time professional employees 
employed in the fall of 1966. All three variables 
are expressed In log form. 



Table I 

STOERRY.X = .18399 
R SQUARED = .94002 
SUM SQR RES =2.43760 
INDVAR USED = 002 





TERM = 


-.68251 




VAR 


COEF 


STD ERR 


T RATIO 


01 


.7470$ 


.03900 


19.15390 


02 


.19289 


.0405$ 


4.75601 



The relevant production function may thus be 
estimated as: 

D=.$053 F l9 $ 75 



As estimated, the IHLs involved exhibit de- 
creasing returns to scale. That Is to say, the sum 
of the coefficients of the function is somewhat less 
than 1. Decreasing returns in the university con- 
text arc subject to an interesting interpretation. So 
long as an institution is small, fairly wide varia- 
tions in the ratio of cnrollces to faculty members 
will not dramatically affect the number of degrees 
granted per cnrollcc (the average productivity of 
cnrollces). However, as the enrollments of the 
institution increase, to reduce the studcnt-facul.y 
ratio to offset the declining average productivity 
of cnrollces requires student-faculty ratios which 
would be exceedingly hard to attain on (he scale 
required. 

Table 2 contains the expected ratio of students 
to degrees (the inverse of the average productivity 
of students) for enrollments at the level of the 
ratio and enrollments at size 30,000. 





Table 2 




Student- 


enrollment 


Enrollment 


Faculty 


at the level 


at Site 


Bello 


of the Bello 


JO, 000 


10/1 


0.52 


5.69 


15/1 


3.90 


e.15 


30/1 


4.64 


/.01 



These ratios may be compared with an "idea 
ratio" of 4.0 for a four-year undergraduate institu 
tion. 

There arc considerable variations in the aver- 
age productivities observed among the 75 institu- 
tions studied. Their inverses range from a low ol 
3.8 to a high of II. with a mean of about 6 
The sources of this variation include program mix 
and (he number of part-time students, as well a< 
the scale of the institutions; however, scale effect! 
do account for a considerable share of the varia- 
tion observed. 

The effects of scale become much more pro- 
nounced if students are distinguished according to 
their status as graduate students, undergraduate 
students, part-time students, and full-time students. 
The estimation of a Cobb-Douglas function which 
included the five factors of production yielded: 

D-2.59 U f 52 O f 13 O p 05 F B Up— .008 
R 2 = .903 
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where Uf=full-lime undergraduates, Gf=full-time 
graduate students, Gp =part*time graduate stu- 
dents, F=full-time professional staff and Up = 

part-time undergraduates. The sum of the coeffic- 
ients is .81, indicating decreasing returns to scale. 
The very small and insignificant coefficient of part- 
time undergraduates suggests that these undergrad- 
uates experience considerable difficulty in com- 
pleting their degrees. 

Two factors, among others, will make difficult 
the development of adequate estimates of IHL 
long-run cost functions. First, the students arc not 
only factors of production, but they provide signifi- 
cant fractions of the demand for degrees. Secondly, 
factor price indexes and cost information by in- 
stitution are r.ol easily obtainable. As an examina- 
tion of state expenditures on higher education 
shows, costs are strongly related to degree produc- 
tion and, thus, indicates that further analyses would 
be worthwhile. Examined by state, 96 per cent of 
the variation in the sum of administrative, instruc- 
tional, maintenance, and library expenditures is 
explained by the total number of degrees produced 
in the states. The estimate, based on 1963-64 ex- 
penditures and degree production in 1967. sug- 
gests that $6,329 in these expen i,l 'ures are made 
for each degree produced. 

The most important implications of these ob- 
servations for the further study of IHLs relate to 
the optimal sire of IHLs and the probable explana- 
tion of their behavior as decision-making units. 
The remainder of the present section will focus on 
institutional sire and the following section will 
briefly examine IHLs as decision-making units. 

The production function estimates, above, pro- 
side frameworks within which to consider univer- 
sity sire as it relates to costs of degree production, 
the demands for unis-ersity outputs, and the idea 
of an optimal sire unis-ersity. The time available 
for the preparation of this paper did not allow a 
full exploration of the abos-e matters. However, 
several preliminary observations can he made. 

The choice of an optimal input mix is not 
fully within the control of an IHL. since the sup- 
ply of student factors and the demand for the 
product are not independent. However, given a 
total budget and a faculty price, there is a degree- 
maximiring level of student enrollment for each 
price which might be placed on the student factor. 

Table 3 coniains calculated cost and productiv- 
ity information for two budget sires ($50,000,000 
ind $100,000,000) and seseral student prices. In 
Older to produce the table, the production func- 
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tion D=.5053F' S' was maximized, subject to 
the constraining budgets for each assumed student 
price. Three features of this table are erf interest. 

While there are decreasing returns to scale evi- 
dent, the computations suggest that the scale of an 
IHL is not nearly so important as the level of the 
student-faculty ratio. Thus, in each case, input can 
be nearly doubled without significantly affecting 
costs per degree or the total output of degrees. On 
the other hand, degree costs rise rapidly as the 
student-faculty ratio is lowered. Optimality then, 
from a strict factor cost standpoint, is to be under- 
stood in terms of the student-faculty ratio and not 
in terms ol (he scale of the institution. If (he sec- 
ond of the two functions estimated above is ex- 
plored. preliminary analyses suggest that scale is of 
sufficiently greater importance as to cause con- 
cern. Unfortunately time did not permit a full ex- 
ploration of the function. 

A reduction in the student-faculty ratio as 
shown in Tabic 3 brings about a concomitant re- 
duction in the number of students per degree. If 
the student's lime Is valued at about $8,000 per 
year, the increase in costs per degree incurred by 
reducing the student-faculty ratio from 22 to I 
to 16 to I is nearly offset. This observation sug- 
gests that an optimal student-faculty ratio ties in 
the neighborhood of 18 to 1. 

"Technology" was defined earlier as a set of 
institutional arrangements by which student and 
faculty inputs are converted into degree outputs. 
The foregoing analysis suggests that at ben the 
structure of the existing arrangements may hold 
degree-seekers in the system for nearly six years. 
The question may be raised as to which institu- 
tional arrangements relate to the extension of 
queues of degree candidates. One's interest in (he 
question is further enhanced on finding that there 
is a strong relationship between student produc- 
tivity and the quality of the product as measured 
by "apples" as a decimal fraction of degrees. In 
Graph 1 the number of "apples" per degree is 
plotted against the numter of cnrollces per degree 
for most of the largest 100 producers of "apples." 
The relationship suggests that the same factors 
which are responsible for increasing the time re- 
quired for a student to complete a degree are re- 
sponsible for the decline in the quality of produc- 
tion. The chief factor leading to declining produc- 
tivity and quality has been related to the tech- 
nology of IHLs. However, the behavior of IHLs 
in internal-resource allocation cannot be dismissed 
as contributing to declining student productivity. 
The market behavior of IHLs will be examined in 
(he following section. 
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Table 3 

C = TOTAL BUDGET 
D = DEGREES GRANTED 
8 = 8TUDENT8 
F = FACULTY 



c 


D 


C/D 


8 


F 


8/D 


8/F 


STUDENT FACTOR PRICE = 


750 










50000000 


631'* 2 


7912.39 


53191.5 


811.171 


8.41744 


65.5737 


100000000 


1212o 


324£ ' 


106383. 


1622.34 


8.77489 


65.5737 


STUDENT FACTOR PRICE r- 


lSGO 










500000 ''O 


3757.42 


13307. 


26595.7 


811.171 


7.07819 


32.7868 


1000000s ) 


7208.72 


13872.1 


53191.5 


1622.34 


7.37877 


32.7868 


STUDENT FACTOR PRICE = 


2250 










50000000 


2772.18 


18036.4 


17730.5 


811.171 


6.39587 


21.8579 


100000000 


5318.5 


18802.3 


35461. 


1622.34 


6.66747 


21.8579 


STUDENT FACTOR PRICE - 


3006 










50000000 


2234.18 


22379.6 


13297.9 


811.171 


5.95203 


16.3934 


100000000 


4286.33 


25330. 


26595.7 


1622.34 


6.20478 


16.3934 


STUDENT FACTOR PRICE ss 


3740 










50000000 


1889.88 


26456.7 


10638.3 


811.171 


5.62908 


13.1147 


100000000 


3625.79 


27680.2 


21276.6 


1622.34 


5.86812 


13.1147 


8TUDENT FACTOR PRICE 


4500 










50000000 


1648.35 


30333.4 


8665.25 


811.171 


5.37826 


10.9289 


100000000 


3162.4 


31621.5 


17730.5 


1622.34 


5.60666 


10.9289 


8TUDENT FACTOR PRICE = 


$240 










50000000 

100000000 


1466.33 

2817.13 


34051.1 

35497.1 


7598.78 

15197.6 


811.171 

1622.34 


5.17494 

5.3947 


9.36767 

9.36767 


STUDENT FACTOR PRICE = 


•000 










50000000 

100000000 


1328.45 

2548.67 


37637.9 

39236.2 


6646.94 

13297.9 


811.171 

1622.34 


5.00504 

5.21758 


8.19671 

8.19671 



Som* Aspects or Life In (ho Markol Pisco 

Inferences about the goals of colleges and uni- 
versities can be made from an examination of the 
work of Allan Cartier and others who have at- 
tempted to determine pecking orders of IHLs 
(Cartter, 1966). By asking faculty, alumni, and 
others and by examining the distribution of awards 
to various institutions, these authors have been 
able to provide rank-ordered lists of institutions 
which are fairly consistent with one another. A 
superior position on these lists is associated with 
such amenities as superior library facilities, higher 
faculty salaries, more Woodrow Wilson fellows, 
and higher freshman entering SAT scores. Studies 
which Ignore the capacity of IHLs to pursue these 
goals seem likely to fail in their attempt to im- 
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prove our understanding of the educational enter- 
prise. 

It was remarked at the beg.nning of the pre- 
vious section that an IffL might be characterized 
as a profit-maximizing institution, insofar as it 
may restrict its output, raise its price, and plough 
back profits from tuition into "apple" production 
and research. 

Table 4 contains the 1963-64 current fund in- 
come for higher education in the 16 states domi- 
nated by the largest 27 “apple" producers. As may 
be noted in Column 3 of that table. 83 per cent 
of the federal research fund income received by 
universities was concentrated in those states. Thus, 
in addition to producing "apples” some consider- 
able share of their output must be thought to be 
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Graph 1. Relationship Between 
"Apples” ?er Degree and Enrollees 
Per Degree for 
Seventy-Five Institutions 




CatoUmi to Fill INI (HEW) 
p*r 0*«rM Product to IMF 



pure and applied research. Thai additional output, 
of course, must be included in the computation of 
their value-added. However, the impact of those 
research activities on student productivity of the 
institutions must be considered. It is not clear that 
in all eases faculty arc able to make these outputs 
complementary. 

Graph 2 illustrates the impact of research-fund 
availability on the composition of production among 
the 27 major Institutions in those states. The esti- 
mated funds available were based on a 10 per cent 
return to the universities' endowments and their 
share of federal research funds as estimated from 
their share of the state output of doctorates. The 
resulting graph suggests that, as higher levels of 
research funds are made available to faculty mem- 
bers. they switch from the teaching of undergradu- 
ates to research and the production of “apples." 
The outlying observations are a special case in 
which the level of state support is a dramatically 
reduced fraction of the total support available to 
the lHLs. 

Oraph 3 shows the relationship between en- 
rollees /faculty member and "apples" 'cnrollcc. 
The 27 institutions under discussion arc circled. It 
may be thought that some of these institutions at- 
tain high levels of "apple" output by holding down 
student-faculty ratios. Among some others, student- 
faculty ratios have not been maintained at suffi- 
ciently low levels to esplain the results. For the 



Graph 2 




Oj ytw p* Iwt thi INI 



Graph .3 
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Table 4. 1963-64 Current Fund Income to Higher Education 



Slat* 


Total Currant 
Fund Income 
(000,090' a) 


Income From 
Federal Ruietrch 
(000, 000'*) 


?Ute 

Government 

(000,000*) 


California 


1,432 


628 


304 


New York 


981 


172 


135 


Illinois 


598 


153 


142 


Massachusetts 


531 


184 


21 


Pennsylvania 


499 


62 


69 


Michigan 


398 


51 


117 


Indiana 


288 


21 


74 


Texas 


360 


32 


111 


Ohio 


381 


36 


58 


Wisconsin 


189 


20 


49 


Connecticut 


132 


20 


19 


Iowa 


172 


17 


48 


New Jersey 


186 


33 


35 


Minnesota 


188 


19 


48 


Washington 


161 


26 


67 


Missouri 


t92 


19 


43 


Sub-total 


6,000 


1,493 


1,240 


U. S. Total 


9,591 


1,797 


2,134 


Sub-total as a 
Percentage of 
U.S. Total 


71 


83 


58 



latter set of institutions, it must be concluded that 
'apples 1 ' are being produced at the expense of pro- 
longing the enrollment of ordinary undergradu- 
ates. Undergraduate productivity at these institu- 
tions also suggests that inurnal student-faculty 
ratios work against undergraduates. Thirteen of the 
27 institutions circled in Graph 3 had more than 
six full-time undergraduates for each baccalaureate 
degree granted in 1967. One institution had more 
than eight full-time undergraduates for each B.A. 
granted and three had between seven and eight. In 
all, these facets of university behavior are strongly 
deserving of further study. 

The Conclusion 

A range of outputs of lHLs has been identified 
and preliminary models tested. In particular, the 
analogy between the form of micro-economic theory 



and lliLs has been considered. While the approach 
seemingly ignores the educational outcome of the 
particular student, it does stress the technological 
limits within which such outcomes are generated. 
It will be necessary to examine both the analogy 
and the data much more closely before reaching 
more than the most tentative conclusions. How- 
ever. several observations are in order. 

First, the model suggests that degree outp at and 
projected degree output (in the case of growing 
IHLs) are useful and important measures of in- 
stitutional sire. Indeed, the use of enrollment fig- 
ures alone may generate highly misleading conclus- 
ions. 

Secondly, the nature of the experience of stu- 
dents. who remain in a baccalaureate program for 
a number of years, should be closely examined in 
order to more fully iihiminate the nature of stu- 
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dent productivity. The “slippage” which appears in 
response to increasing student-faculty ratios may 
have little to do with the content of the program. 
That is, a kind of reverse “Hawthorne effect” may 
be responsible for declining productivity. If such 
an effect is operant, significant increases in the 
student-faculty ratio as a part of “open enroll- 
ment” programs and the like would only serve to 
increase campus tensions. It must be noted in this 
regard that there are few, if any, institutions with 
low average entering freshman SAT scores and 
low student-faculty ratios. The final outcome of an 
experiment which dramatically changes IHLs in 
this regard is by no means dear. 

Finally, it is clear that the student demand for 
degrees must be carefully studied. The time cost 
of choosing an institution with a high student-fac- 
ulty ratio can be considerable. The job market 
would surely value the time of able students dif- 
ferently than the time of their less able counter- 
parts. A failure to consider student reactions to 
the counter-pulls of marKet and social forces could 
result in difficulty for IHLs. 
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“Decisions with respect to graduate education may be made 
at various levels. There are decisions at the national and 
state levels concerning the establishment ot new Institutions 
and new programs and the expansion or elimination of exist- 
ing programs. Should a new Ph.D. program In African Studies 
be approved or a new medical school established? Individual 
Institutions must choose between alternative strategies and 
alternative programs. Can a university justify its use ot re- 
sources In Astronomy or Southeast Aslan Studies, given the 
outputs, costs, and alternative demands tor scarce re- 
sources?" 

"Was the post-war emphasis In the United States on science 
and on the training of college teachers with the Ph.D. at the 
expense of the professions a wise allocation of resources?" 

“The completion of some degrees, such as the LL.B., M.D., or 
Ph.D., may have more significance than the completion of 
the M.A. or M.S. degrees, since In some Institutions the latter 
degrees are awarded as a consolation prize or as a step on 
the road to a Ph.D." 
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Outputs of Higher Education: 
Graduate Education 

JOHN PERRY MILLER 



The Task 

The assigned task is to define the processes and 
products of graduate education in significant and 
operationally measurable terms. The stated motive 
is the hope to improve 41 . , . decisions governing 
the allocation of resources in colleges and univer- 
sities.” i.e., to identify better choices between the 
alternatives open to decision-makers. VYc are “ . . . 
urged to consider the outputs of higher education 

in the broadest sense No attempt to limit the 

characterization of outputs to the economic sphere 
alone is either intended or desired.” Moreover, 
while we are asked to concentrate on the ”... out- 
put of the university viewed as an entity, rather 
than , , , each department or other small unit,” 
we were not explicitly confined to identifying 
“final” outputs or precluded from measurement of 
"intermediate products.” 

Perspective 

The objectives of our educational system are 
many, They include the development of values and 
character and the general socialization of the in- 
dividual, in addition to the acquisition and dissem- 
ination of knowledge and the education of man- 
power with advanced and professional skills. The 
outputs are, consequently, multiple and generally 
“joint-products” in the economic sense. In the pres- 
ent state of knowledge, it is questionable whether 
objective and meaningful measures can be devel- 
oped for some of the important outputs of higher 
education. Therefore, responsible decision-makers 
must rely in part upon their judgment and that of 
others concerning the reality and importance of 
some such outputs and weigh these in the balance 
with other outputs that are subject to more objec- 
tive and meaningful measures. Judgment, intui- 
tion, and faith play an important part in most 
decisions, both private and public, but they are 
especially important in the case of decisions con- 
cerning education in which “public” benefits, in 
contrast to private benefits, bulk so large. 

It is important to recognize that the develop- 
ment of some types of operational measures of the 
output of education may have effects on the educa- 
tional process and therefore upon the outputs them- 
selves. For example, nationwide or statewide tests 



may become ends in themselves and therefore bias 
or warp the educational process, if they are thought 
to have consequences for the students, teachers, or 
administrators. Not infrequently, students arc 
trained to do well on the tests at the expense of 
other values of education that are not measured by 
the test, because the student wishes admission to 
further study or the teacher gains recognition by 
the record of his students on the tests. Such prob- 
lems may arise, not so much from the nature of the 
measure as from its perceived use. The problem of 
evaluation is one of defining the objectives of the 
educational program and devising systems of evalu- 
ation that determine the extent to which the objec- 
tive is attained, while reinforcing the incentives 
to pursue the desired ends. For example, a sys- 
tem of budgeting that is tied rigidly to the num- 
ber of credit uours may provide an incentive to 
increase enrollment. Or a system based on degrees 
awarded may affect the level of achievement re- 
quired for the award of degrees. Doth systems favor 
“popular” fields of study and work to the disad- 
vantage of the less popular or more “exotic” areas 
of study. 

The danger that these measures may be coun- 
ter-productive must be weighed against the disad- 
vantages of having no measures. The resources 
available to education are necessarily limited, and 
higher education must compete with other claims 
upon limited resources. Decisions will be made at 
the levels of the nation, state, educational system, 
and college or university. Decisions based upon 
knowledge, including knowledge reflected in meas- 
ures of output, made by persons of good will and 
good judgment, are likely t be better than decisions 
made in the absence of knowledge. Such knowledge 
may, of course, be misused by persons of ill will 
or bad judgment. But this is a problem not for 
those concerned with devising measures, but for 
those responsible for establishing and staffing the 
decision-making process, i.e., those who use the 
measures. Knowledge without a rational decision 
process, staffed by persons of integrity, gains us lit- 
tle, Conversely, a rational decision-making process, 
appropriately staffed, presupposes relevant knowl- 
edge. 

Decisions with respect to graduate education 
may be made at various levels. There are decisions 
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at the national and state levels concerning the es- 
tablishment of new institutions and new programs 
and the expansion or elimination of existing pro- 
grams. Should a new Ph.D. program in African 
Studies be approved or a new medical school es- 
tablished? Individual institutions must choose be- 
tween alternative strategies and alternative pro- 
grams. Can a university justify its use of resources 
in Astronomy or Southeast Asian Studies, given 
the outputs, costs, and the alternative demands 
for scarce resources? Such incremental decisions 
must be made periodically, whether consciously or 
unconsciously and with or without relevant data. 
They presuppose, of course, some sense of insti- 
tutional objective, some preference system, in 
terms of which choices can be made in light of the 
perceived alternatives. These preference systems 
will vary between institutions, and the parties at 
interest in ary particular institution or in any 
given decision may operate with different prefer- 
ence systems. The outcome will depend, then, not 
only on the perceived alternatives and objectives 
or preference systems, but also on the collective 
decision process. In light of the diversity of edu- 
cational objectives and preference systems and the 
variety of decision processes, a multiplicity of 
measures of output takes on special significance. 

The importance of an incremental approach to 
improved decision-making should not be underes- 
timated. There are many operational decisions in 
an individual institution that can be the subject of 
improved decision. In the battle for resources at 
the macro-level, evidence that decisions at the 
micro-level are based on careful analysis of relevant 
information and hard choices between alternatives, 
may be decisive. 

Finally, since decisions are decentralized, it will 
often be useful to measure intermediate outputs, 

i.e., to identify points in the process of graduate 
education where changes in the inputs can have 
significant elects on the efficiency of the process. 
Thus, a decision might be made to change the al- 
location of resources between recruitment, selec- 
tion, remedial education, and the normal education 
required for the degree. For example, “dropouts” 
may be reduced by better selection processes or 
remedial education. But at what cost? 



Function of Graduate Education 

By graduate education, I mean all postbacca- 
laureate education in the arts and sciences and the 
professions, including postdoctoral and continuing 
education. 



I assume that the principal purposes of gradu- 
ate education so defined are: 

1. To add to the body of knowledge through 
research. 

2. To disseminate the body of knowledge. 

3. To increase the professional effectiveness 
of those participating in graduate education, 
including faculty and students. (This is 
clearly true of training for the Ph.D. and 
the recognized professional degrees; it is 
also true of much education of masters* 
candidates and of much continuing educa- 
tion.) 

I assume that the basic elements of a student’s 
character and V 2 1 es, and his preparation for citi- 
zenship are formed prior to embarking on gradu- 
ate education and, consequently, that the measure- 
ment of such social benefits of graduate education 
is not a significant problem. To be sure, graduate 
education may strengthen or reinf^ r ce these quali- 
ties. And some students look upon graduate edu- 
cation, especially the first year, as simply an ex- 
tension of the type of general education character- 
istic of the undergraduate college. Others conceive 
it as a period for testing out their interest in some 
line of graduate or professional work. But I see 
no reason, apart from socialization of a graduate 
student to the norms of his intended profession, to 
assume that such social benefits associated with a 
year of graduate study will, on the average, differ 
from those associated with alternative experiences 
such as a year of “employment.” This is not to 
deny that there are social benefits other than the 
professional competence acquired that are asso- 
ciated with graduate education. But I believe that 
these benefits are subject to eventual diminishing 
returns in education as elsewhere and that, for 
most, the rate of return in terms of the social bene- 
fits in question will be low by the time of graduate 
study. Moreover, other uses of one’s time, for ex- 
ample, employment in the world of affairs, offer 
similar benefits in the development of character, 
values, and preparation for citizenship — benefits 
which may be even greater than those from addi- 
tional years of study because of the qualitative 
change in perspective. 

The fact that these “social benefits” of the type 
indicated, although real, are not the primary ef- 
fect of graduate education, should simplify the 
problem of measurement and decision in this area. 

Bui while the principal purposes of graduate 
education are the search for new knowledge, the 
dissemination of knowledge, and the development 
of professional competence, decisions are often 
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affected by other objectives arising from social or 
political considerations. For example, the goal of 
“equal opportunity” directs attention to the recruit- 
ment and education of various minority groups and 
of women. The mix of students, admitted and 
educated, may have effects on inputs (e.g., the cost 
of remedial education for the culturally deprived) 
and on the final outputs however measured in vari- 
ous ways. At times, national policy with respect to 
the education of foreign students has led to simi- 
lar special objectives in admission. Likewise, con- 
cern for the geographical distribution of education- 
al opportunities and research facilities may also dic- 
tate special attention to the geographical dispersion 
of facilities for graduate education. 

Process of Graduate Education 

The process of graduate education may be 
classified under seven headings: 

1. Recruitment and selection of students 

2. Assembling of faculty 

3. Acquisition and administration of resources, 
other than faculty, necessary for education 
and research 

4. Education of the students (graduate, post- 
doctoral, and in-career students), including 
remedial education 

5. Research 

6. Provision of services to the community 
(e.g., practice teaching, internships, con- 
sultation) 

7. Placement and replacement of students 

There are several perspectives from which de- 
cisions concerning resource allocation to graduate 
education may be made. One can consider the al- 
location to graduate education as a whole or to 
particular systems or institutions. One can focus 
on functional sub-units such as different disciplines 
or professions; different levels of study such as 
masters, predoctoral, postdoctoral, and in-career 
study; or different stages in the process of gradu- 
ate education, e.g., recruitment, selection, remedial 
work, education, or placement. It is relevant to 
consider how resources should be allocated be- 
tween different disciplines and professions or be- 
tween different degree programs. For example, 
was the postwar emphasis in the United Slates on 
science and on the training of college teachers 
with the Ph.D. at the expense of the support of the 
professions a wise allocation of resources? Is the 
current allocation of resources in medicine be- 
tween research and teaching appropriate? Should 
the large prestigious universities devote so many 
resources to the search for talent of all sorts? Or 



only to search for talent among minority groups? 
Is the talent search of the Woodrow Wilson Fellow- 
ship Program worth the cost? 

The variety of questions concerning the use 
of resources in higher education emphasizes again 
the need for a pluralistic definition of outputs and 
a complex system of measurement. 

Outputs I: Value-added 

The first instinct of an economist is to explore 
the concept of “value-added” by graduate educa- 
tion. This is, of course, a useful measure for judg- 
ing the overall effectiveness of the total system of 
graduate education and of many subdivisions. It 
has the advantage of being expressed in the com- 
mon denominator of money and, therefore, can 
be readily balanced against costs, which are to a 
large extent expressed in such f erms. 

But t!:e value-added concept is not without dif- 
ficulties. A good first approximation to value-added 
may be obtained by measuring the increase in the 
capitalized value of the income stream for those 
attaining a given level of graduate education over 
that of a control group of persons of similar prior 
achievement and potential who do not go on to 
graduate education. The control group might be 
a group who have completed the bachelor's degree 
with similar predictive scores for success in a 
given graduate program. The development of ap- 
propriate predictive scores does, of course, present 
difficulties. But standardized tests used for admis- 
sion tu graduate study in various disciplines and 
professions are useful in conjunction with cor- 
rected undergraduate grade records. But can we 
assume that groups with a similar distribution of 
test scores and grade averages have the same prob- 
abilities of performance? Is it not probable that the 
decisions to undertake or not to undertake graduate 
stuJy reflect differences in motivations, values, or 
character that will affect earning power whether 
one undertakes graduate study or not? If so, whit 
kind of correction would be necessary? It is con- 
ceivable that in some fields those who do not go 
on to graduate study may have less ambition and 
drive than those who do; if so, the value-added 
may be oversrated. In other fields, those who un- 
dertake graduite study may be less able to cope 
with the world without such education than the 
control group; if so, without graduate education 
they might have had a lesser income than the con- 
trol group; thus, the value-added may be under- 
stated. Can the significance of these possibilities 
be determined and, if they are significant, can we 
correct for them? 
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But the value-added accruing to the individual 
by way of money income may not be all the value- 
added. There are some free services rendered inci- 
dental to graduate study, e.g., legal aid or the 
services of interns. Moreover, many people render 
free public service throughout their lives (com- 
munity and professional organizations* volunteer 
workers, etc.). Is the value of these free services 
rendered by those with graduate education greater 
or less than those rendered by people in the con- 
trol group? If they are, the value-added by gradu- 
ate education should be adjusted accordingly. It 
should not be impossible to make a sample survey 
that would throw light on this issue. 

And what of the value-added by new knowl- 
edge resulting from research? If the knowledge is 
patented or copyrighted, the value-added as meas- 
ured by the market will reflect the market value 
of the research. But if it is disseminated widely at 
no cost to those who benefit from it, the benefits 
will not be included in the private value-added. 
Can some allowance be made for these public bene- 
fits? This problem is not serious in some of the 
professional schools where research plays a lesser 
role than in the basic arts and sciences. Moreover, 
if the costs of teaching and research could be sep- 
arated, the problem might be avoided. However, at 
the level of graduate education, teaching and re- 
search are inextricably entwined, and attempts to 
allocate costs are inevitably arbitrary. The prob- 
lem remains. 

The value-added as defined above can be used 
to measure output for various disciplines and pro- 
fessions on a nationwide, regional, or individual 
institutional basis. Likewise, this measure of out- 
put can be applied to groups classified by the num- 
ber of years of study, degree obtained, sex, or by 
national, racial, or religious origin. The estimates 
are probably more accurate for the professional 
schools than for the graduate schools of arts and 
sciences because of the lesser role of research. 

Outputs II: Other Measures 

There are some other non-monetary measures 
of the outputs of the process of graduate education 
that are relevant to resource allocation. 

Obvious gross measures of output are the num- 
ber of man-years of study and number receiving 
different degrees. Man-years of study makes allow- 
ance for the education of those who do not com- 
plete the degree or who are enrolled for non-degree 
programs. Such education has value, and it is often 
a matter of conscious decision to provide it. De- 



grees granted are an obvious measure which may 
be particularly useful in allocating resources 
within an institution or in comparing the perform- 
ance of similar institutions. But the measure is of 
limited use if the mix of degree and non-degree 
students educated jointly differs substantially. 

Concern for “equality of opportunity” to par- 
ticipate in graduate education and in the life-styles 
and careers to which it leads, suggests an analysis 
of data on recruitment, admission, matriculation, 
and completion of the degree for various student 
population classified by sex, ethnic origin, religion, 
and geographical origin. Thus, it may be useful to 
compare the proportion of female baccalaureates 
applying to graduate school with that for men of 
the same potential, or black baccalaureates with 
white. A similar comparison of the ratio of offers 
of admission to applications may be useful as may 
the ratio of (hose receiving a graduate degree to 
the number matriculating or to the number receiv- 
ing a baccalaureate. Each of these comparisons 
would throw some light on opportunity differen- 
tials in graduate education. The political and social 
significance of equalizing opportunities suggest the 
urgency of careful analysis of the flow of students 
of these various categories. 

A comparative analysis of applications for vari- 
ous schools or an analysis of differences in the 
trends in the number and composition of applica- 
tions may throw light on the effectiveness of re- 
cruitment or upon perceptions of a school's excel- 
lence. Such data must be interpreted with care, 
since applicants may be affected by such factors 
as changes in or differences in application fees, fi- 
nancial aid, and draft policies. But recruitment and 
selection of students involve costs. At some cost, 
more applications may be generated and better 
selection can be made. And at some cost in fi- 
nancial aid and other inducements, the rate of 
acceptance of offers can be improved. How should 
resources be allocated? To improve decisions con- 
cerning the allocation of resources, whether within 
an institution or on a national or state basis, in- 
volves better choices between expenditures on re- 
cruitment, selection, financial aid, faculty, and other 
resources (including housing and social facilities). 
This requires information concerning the respon- 
siveness of certain intermediate outputs to inputs. 

The analysis of those matriculating who com- 
plete their degree (and its reverse, the dropout 
rate) is another useful measure of output. Clearly 
the completion of a degree is only a measure of 
academic standards met. The significance varies 
between degrees, universities, departments, and in- 
dividuals. Moreover, it will be affected by a variety 
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of factors such as the quality of matriculants, fi- 
nancial aid policy, standards of achievement re- 
quired for the degree, as well as the effectiveness 
of the educational program, But trends and differ- 
ences in the rate of completion of the degree may 
be relevant tc the allocation of resources. The com- 
pletion of some degrees, such as the LL.B., M,D,, 
or Ph.D., may have more significance than the com- 
pletion of an M.A. or M.S. degree, since in some 
institutions the latter degrees are awarded as a 
consolation prize or as a step on the road to the 
Ph.D. Many proceeding to the Ph.D. do not seek 
the masters degree, especially if a special fee is as- 
sessed. But if interpreted with care, statistics on 
the percentage of those completing various de- 
grees will be relevant to decisions concerning the 
allocation of resources to graduate education. 

The same is true of studies of the length of 
time required to complete the graduate degree. 
While many studies, especially those concerned 
with the Ph.D., have measured the length of time 
from the completion of the bachelor’s degree to 
the completion of the advanced degree, for pres- 
ent purposes the time from the commencement of 
graduate study to completion of the degree or the 
years of registration may be more relevant, But 
there are real difficulties in this problem in the 
case of students engaged in part-time study and 
in the case of candidates for the Ph.D., because 
of the stretchout in the process of completing the 
dissertation while fully or partially employed. Dif- 
ferences in practices concerning registration, fi- 
nancial aid, and rules concerning work in absentia 
confuse the meaning of these data, especially in the 
case of Ph.D, candidates. 

Alumni Experience Indicative 

An analysis of the employment experience of 
alumni may also be indicative of output. Alternate 
types of employment can bo distinguished (teach- 
ing, research, government, business, professions, 
etc.) and differences and trends analyzed. Clearly 
tlnse data must be analyzed with care, since they 



are affected not only by the quality of matriculants 
and the effectiveness of the educational process, 
but also by the changes in employment opportuni- 
ties and the effectiveness of placement efforts. 

Finally, in the matter of research some indices 
of differences and change are available. Data on 
the number of publications and the frequency of 
citations ave available or can be devised. White 
these may not indicate whether the resources allo- 
cated to research are justified by the results, they 
may be useful for comparing the research effective- 
ness of similar departments in different institu- 
tions. It is true that research and publication are 
not always synonymous so that these measures are 
of limited value in comparing individuals. But our 
interest is not in micro decisions concerning the 
appointment of an individual but rather in meas- 
ures as a guide to the allocation of resources at the 
departmental or higher level in situations where 
research is one of the valued outputs. 

Each of these various quantitative measures of 
performance has serious limitations, some of which 
have been mentioned. Moreover, they do not pro- 
vide a single valued measure of output, achieve- 
ment, or effectiveness. But from them can be con- 
structed a view of trends in the performance of 
various sectors of the world of graduate education 
or differences between sectors or institutions. The 
resulting view can be related to differences in the 
level or type of input. Such information can be use- 
ful at several levels in the effort to make better, 
although not necessarily the best possible, decisions. 
One advantage of such data is that they can be col- 
lected as part of the administrative routine of our 
graduate schools so that they can be kept current 
at little expense. In this respect they differ from 
data on value-added which are expensive to con- 
struct and difficult to keep current; moreover, they 
involve difficult projections of the future if they 
are to be relevant to recent performance. Another 
advantage of these data generated by the admin- 
istrative process is that their generation need not 
affect the educational process although their use 
not only may, but it is hoped, will. 
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AN ACCOUNTING STRUCTURE FOR THE 
OUTPUTS OF HIGHER EDUCATION: 
ONE PROPOSAL 



Draft for Review Purposes Only 
Instructional Outputs 



Variables 

Cognitive Attributes of Students: 

Level of General Knowledge 
Level of Knowledge In Chosen Field 
Basic Language Arts Skills 
Critical Thinking and Reasoning 
General Intelligence 



Source of Measures 



Test Scores 
Test Scores 
Test Scores 
Test Scores 
Test Scores 



Affective Attributes of Students: 

Self-concept 

Satisfaction with Educational Experience 

Citizenship 

Values 

Achievement Motivation 



Questionnaire Responses 
Questionnaire Responses 
Questionnaire Responses 
Questionnaire Responses 
Questionnaire Responses 



Tangible Attributes of Student*: 

Earning Power 

Awards 

Affiliations 

Avocations 

G,P,A. 

Level of Educational Attainment 
Flexibility of Employment 
Areas of Career Interest 



Placement and Employment Data 
Number and Stature of Awards 
Number and Kind of Affiliations 
Number and Kind of Hobbles 
Academic Record Data 
Academic Record Data 
Placement and Employment Data 
Questionnaire Responses 



Institutional Environment Outputs 



Variables 

Academic Environment Attributes: 

Rate of Student Success 
Mean Time to Reach Degree 
Facully Turnover 
Faculty Availability to Students 
Academic Resources Available 
Quality of Instruction 
Academic Aptitude Mix 
Student Stress 
Faculty Stress 

Social Environment Attributes: 

Degree of Social Activity on Campus 
Racial Mix 
Socio-Economic Mix 
Family Attitude Characteristics 
Social Involvement of Student Body 
Per cent Resident (on campus) Students 
Rate of Marriage Among Students 
PhysfcaJ Environment 



Source of Measures 



Dropout Data 
Student Record Data 
Faculty Record Data 
Student Questionnaire 
Library Data 

Faculty & Student Questionnaire 
Entering Student SAT Scores 
Student Questionnaire 
Faculty Questionnaire 



Activity Records and Questionnaire 
Student & Faculty Rocords 
Student Records 
Questionnaire 
Questionnaire 

Housing and Student Records 
Student Records 

Physical Plant Data and Questionnaire 
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Research Outputs 



Variables 



Source of Measures 



Reorganization of Knowledge 

New Inventions and Developments 
{Applied Research Products; 

New Ideas and Concepts 
(Pure Research Outputs) 

Personal Involvement of Students 
and Others (Instructional spinoff) 



Number of new books, textbooks, etc, 

Number of patents, adopted procedures, etc, 

Number of articles, papers, awards, citations, etc. 

Number of hours involvement on projects by 
students, Industry, personnel, etc. 



Pubtlc Service Output* 



Variables 



Source of Measures 



Student Involvement In Community 
Faculty Involvement In Community 
Cultural Activities Available 
Recreation Activities Available 
Continuing Education Activities 

Social Criticism 
Personal Services 
Indirect Community Benefits 
Community Psychic Income 



Hours of time, type of project, questionnaire 

Hours of time, type of project, questlonnalro 

Number, type, duration, attendance, participation 

Number, type, duration, attendance, participation 

Number, type, duration, enrollment, quality, and 
satisfaction, questionnaire 

Amount, frequency, Intensity, effects of confrontation 

— Students and Community 

- Faculty and Community 

Number of health care patients, counseling patients, 
psychological testing, legal advice requests, etc. 
(dollar value of such services) 

Students available as employees, drawing power of 
the community as a place of residence for 
professional and skilled persons 

Public pride, awareness that expertise is available 
if needed 



Product Testing 



Number and types of products and materials 
tested for government and industry 
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The first section includes those references the authors felt were germane to 
their topics, as well as other references that deal with specific output variables 
for their measures. Certain models and analyses of educational outputs inherently 
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The third list contains references dealing primarily with educational cost. 
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